2023-01-25T06:39:01.109 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T06:39:01.110 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} --archive /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136940 --name yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi --verbose --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136940/orig.config.yaml 2023-01-25T06:39:01.141 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T06:39:01.228 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136940 branch: wip-yuri-testing-2023-01-23-0926 description: rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} email: yweinstein@radhat.com first_in_suite: false job_id: '7136940' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri-testing-2023-01-23-0926 ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug verify cached snaps: true osd debug verify missing on start: true osd mclock override recovery settings: true osd op queue: debug_random osd op queue cut off: debug_random flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: b0d2592529629ab4f86fed5991e556a87dcda563 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: b0d2592529629ab4f86fed5991e556a87dcda563 workunit: branch: wip-yuri-testing-2023-01-23-0926 sha1: b0d2592529629ab4f86fed5991e556a87dcda563 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - osd.0 - osd.1 - osd.2 - client.0 - client.1 sha1: b0d2592529629ab4f86fed5991e556a87dcda563 sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri-testing-2023-01-23-0926 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b0d2592529629ab4f86fed5991e556a87dcda563 targets: smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: conf: global: debug client: 20 debug mds: 20 debug ms: 1 osd max object name len: 460 osd max object namespace len: 64 log-ignorelist: - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - exec: client.0: - ceph osd pool create data_cache 4 - ceph osd tier add cephfs_data data_cache - ceph osd tier cache-mode data_cache writeback - ceph osd tier set-overlay cephfs_data data_cache - ceph osd pool set data_cache hit_set_type bloom - ceph osd pool set data_cache hit_set_count 8 - ceph osd pool set data_cache hit_set_period 3600 - ceph osd pool set data_cache min_read_recency_for_promote 0 - ceph-fuse: null - exec: client.0: - sudo chmod 777 $TESTDIR/mnt.0/ - dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5 - ls -al $TESTDIR/mnt.0/foo - truncate --size 0 $TESTDIR/mnt.0/foo - ls -al $TESTDIR/mnt.0/foo - dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5 - ls -al $TESTDIR/mnt.0/foo - cp $TESTDIR/mnt.0/foo /tmp/foo - sync - rados -p data_cache ls - - sleep 10 - rados -p data_cache ls - - rados -p data_cache cache-flush-evict-all - rados -p data_cache ls - - sleep 1 - exec: client.1: - hexdump -C /tmp/foo | head - hexdump -C $TESTDIR/mnt.1/foo | head - cmp $TESTDIR/mnt.1/foo /tmp/foo teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:20:59 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T06:39:01.228 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa; will attempt to use it 2023-01-25T06:39:01.229 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa/tasks 2023-01-25T06:39:01.229 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T06:39:01.231 INFO:teuthology.task.internal:Checking packages... 2023-01-25T06:39:01.258 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'b0d2592529629ab4f86fed5991e556a87dcda563' 2023-01-25T06:39:01.258 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T06:39:01.259 INFO:teuthology.packaging:ref: None 2023-01-25T06:39:01.259 INFO:teuthology.packaging:tag: None 2023-01-25T06:39:01.260 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-01-23-0926 2023-01-25T06:39:01.260 INFO:teuthology.packaging:sha1: b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T06:39:01.260 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri-testing-2023-01-23-0926 2023-01-25T06:39:01.411 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1959-gb0d25925-1focal 2023-01-25T06:39:01.411 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T06:39:01.417 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T06:39:01.417 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T06:39:01.456 INFO:teuthology.task.internal:Saving configuration 2023-01-25T06:39:01.468 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T06:39:01.484 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T06:39:01.514 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136940', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 06:33:05.650350', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-01-25T06:39:01.514 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T06:39:01.520 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1'] 2023-01-25T06:39:01.521 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T06:39:01.583 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f47847655e0>, signals=[15]) 2023-01-25T06:39:01.585 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T06:39:01.591 INFO:teuthology.task.internal:Opening connections... 2023-01-25T06:39:01.591 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2023-01-25T06:39:01.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:39:01.641 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T06:39:01.648 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2023-01-25T06:39:01.762 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2023-01-25T06:39:01.763 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2023-01-25T06:39:01.769 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="Ubuntu" 2023-01-25T06:39:01.769 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-01-25T06:39:01.769 INFO:teuthology.orchestra.run.smithi117.stdout:ID=ubuntu 2023-01-25T06:39:01.769 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE=debian 2023-01-25T06:39:01.770 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-01-25T06:39:01.770 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="20.04" 2023-01-25T06:39:01.770 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.ubuntu.com/" 2023-01-25T06:39:01.770 INFO:teuthology.orchestra.run.smithi117.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-01-25T06:39:01.770 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-01-25T06:39:01.771 INFO:teuthology.orchestra.run.smithi117.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-01-25T06:39:01.771 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_CODENAME=focal 2023-01-25T06:39:01.771 INFO:teuthology.orchestra.run.smithi117.stdout:UBUNTU_CODENAME=focal 2023-01-25T06:39:01.772 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2023-01-25T06:39:01.791 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T06:39:01.800 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T06:39:01.823 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T06:39:01.824 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2023-01-25T06:39:01.830 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T06:39:01.849 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T06:39:01.849 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2023-01-25T06:39:01.857 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T06:39:01.903 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T06:39:01.917 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T06:39:01.918 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T06:39:01.918 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T06:39:01.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T06:39:01.924 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2023-01-25T06:39:01.931 INFO:teuthology.orchestra.run.smithi117.stdout:5.4.0-124-generic 2023-01-25T06:39:01.931 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.4.0-124-generic 2023-01-25T06:39:01.931 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get clean 2023-01-25T06:39:02.032 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2023-01-25T06:39:02.343 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-01-25T06:39:02.344 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-01-25T06:39:02.405 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-01-25T06:39:02.787 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-01-25T06:39:02.868 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,960 kB] 2023-01-25T06:39:03.012 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,341 kB] 2023-01-25T06:39:03.030 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [543 kB] 2023-01-25T06:39:03.055 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [320 kB] 2023-01-25T06:39:03.077 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [11.7 kB] 2023-01-25T06:39:03.094 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [29.1 kB] 2023-01-25T06:39:03.112 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,463 kB] 2023-01-25T06:39:03.146 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [207 kB] 2023-01-25T06:39:03.166 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [624 B] 2023-01-25T06:39:03.183 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [576 kB] 2023-01-25T06:39:03.209 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [787 kB] 2023-01-25T06:39:03.235 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [152 kB] 2023-01-25T06:39:03.254 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [16.9 kB] 2023-01-25T06:39:03.494 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [780 kB] 2023-01-25T06:39:03.514 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [404 kB] 2023-01-25T06:39:03.524 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.2 kB] 2023-01-25T06:39:03.525 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [30.4 kB] 2023-01-25T06:39:03.526 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,560 kB] 2023-01-25T06:39:03.594 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [220 kB] 2023-01-25T06:39:03.599 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [620 B] 2023-01-25T06:39:03.600 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [708 kB] 2023-01-25T06:39:03.618 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,019 kB] 2023-01-25T06:39:03.644 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [236 kB] 2023-01-25T06:39:03.649 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [23.4 kB] 2023-01-25T06:39:03.691 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.2 kB] 2023-01-25T06:39:03.691 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,408 B] 2023-01-25T06:39:03.692 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [604 B] 2023-01-25T06:39:03.692 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-01-25T06:39:03.693 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.9 kB] 2023-01-25T06:39:03.698 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-01-25T06:39:06.430 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 13.8 MB in 2s (5,678 kB/s) 2023-01-25T06:39:07.762 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2023-01-25T06:39:07.788 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-01-25T06:39:07.861 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2023-01-25T06:39:08.049 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2023-01-25T06:39:08.050 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2023-01-25T06:39:08.212 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2023-01-25T06:39:08.213 INFO:teuthology.orchestra.run.smithi117.stdout: linux-generic linux-headers-5.4.0-137 linux-headers-5.4.0-137-generic 2023-01-25T06:39:08.213 INFO:teuthology.orchestra.run.smithi117.stdout: linux-headers-generic linux-image-5.4.0-137-generic 2023-01-25T06:39:08.213 INFO:teuthology.orchestra.run.smithi117.stdout: linux-modules-5.4.0-137-generic linux-modules-extra-5.4.0-137-generic 2023-01-25T06:39:08.214 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2023-01-25T06:39:08.214 INFO:teuthology.orchestra.run.smithi117.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-01-25T06:39:08.214 INFO:teuthology.orchestra.run.smithi117.stdout:Recommended packages: 2023-01-25T06:39:08.215 INFO:teuthology.orchestra.run.smithi117.stdout: thermald 2023-01-25T06:39:08.266 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2023-01-25T06:39:08.266 INFO:teuthology.orchestra.run.smithi117.stdout: linux-headers-5.4.0-137 linux-headers-5.4.0-137-generic 2023-01-25T06:39:08.266 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.4.0-137-generic linux-modules-5.4.0-137-generic 2023-01-25T06:39:08.267 INFO:teuthology.orchestra.run.smithi117.stdout: linux-modules-extra-5.4.0-137-generic 2023-01-25T06:39:08.267 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages will be upgraded: 2023-01-25T06:39:08.268 INFO:teuthology.orchestra.run.smithi117.stdout: linux-generic linux-headers-generic linux-image-generic 2023-01-25T06:39:08.321 INFO:teuthology.orchestra.run.smithi117.stdout:3 upgraded, 5 newly installed, 0 to remove and 129 not upgraded. 2023-01-25T06:39:08.321 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 77.1 MB of archives. 2023-01-25T06:39:08.321 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 380 MB of additional disk space will be used. 2023-01-25T06:39:08.321 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-137-generic amd64 5.4.0-137.154 [15.0 MB] 2023-01-25T06:39:08.663 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-137-generic amd64 5.4.0-137.154 [10.5 MB] 2023-01-25T06:39:08.754 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-137-generic amd64 5.4.0-137.154 [39.2 MB] 2023-01-25T06:39:09.215 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.137.135 [1,896 B] 2023-01-25T06:39:09.215 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.137.135 [2,528 B] 2023-01-25T06:39:09.216 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-137 all 5.4.0-137.154 [11.0 MB] 2023-01-25T06:39:09.307 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-137-generic amd64 5.4.0-137.154 [1,400 kB] 2023-01-25T06:39:09.317 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.137.135 [2,392 B] 2023-01-25T06:39:10.002 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 77.1 MB in 1s (73.9 MB/s) 2023-01-25T06:39:10.222 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-modules-5.4.0-137-generic. 2023-01-25T06:39:11.975 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-01-25T06:39:11.979 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../0-linux-modules-5.4.0-137-generic_5.4.0-137.154_amd64.deb ... 2023-01-25T06:39:12.069 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-modules-5.4.0-137-generic (5.4.0-137.154) ... 2023-01-25T06:39:14.035 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-image-5.4.0-137-generic. 2023-01-25T06:39:14.052 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../1-linux-image-5.4.0-137-generic_5.4.0-137.154_amd64.deb ... 2023-01-25T06:39:14.148 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-image-5.4.0-137-generic (5.4.0-137.154) ... 2023-01-25T06:39:15.304 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-137-generic. 2023-01-25T06:39:15.317 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-137-generic_5.4.0-137.154_amd64.deb ... 2023-01-25T06:39:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-modules-extra-5.4.0-137-generic (5.4.0-137.154) ... 2023-01-25T06:39:20.081 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../3-linux-generic_5.4.0.137.135_amd64.deb ... 2023-01-25T06:39:20.216 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-generic (5.4.0.137.135) over (5.4.0.124.125) ... 2023-01-25T06:39:20.541 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.137.135_amd64.deb ... 2023-01-25T06:39:20.693 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-image-generic (5.4.0.137.135) over (5.4.0.124.125) ... 2023-01-25T06:39:20.996 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-headers-5.4.0-137. 2023-01-25T06:39:21.011 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../5-linux-headers-5.4.0-137_5.4.0-137.154_all.deb ... 2023-01-25T06:39:21.046 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-5.4.0-137 (5.4.0-137.154) ... 2023-01-25T06:39:26.447 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-headers-5.4.0-137-generic. 2023-01-25T06:39:26.468 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../6-linux-headers-5.4.0-137-generic_5.4.0-137.154_amd64.deb ... 2023-01-25T06:39:26.513 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-5.4.0-137-generic (5.4.0-137.154) ... 2023-01-25T06:39:29.123 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.137.135_amd64.deb ... 2023-01-25T06:39:29.243 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-generic (5.4.0.137.135) over (5.4.0.124.125) ... 2023-01-25T06:39:29.568 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-5.4.0-137 (5.4.0-137.154) ... 2023-01-25T06:39:29.697 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-modules-5.4.0-137-generic (5.4.0-137.154) ... 2023-01-25T06:39:30.916 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-5.4.0-137-generic (5.4.0-137.154) ... 2023-01-25T06:39:31.042 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-image-5.4.0-137-generic (5.4.0-137.154) ... 2023-01-25T06:39:32.201 INFO:teuthology.orchestra.run.smithi117.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-137-generic 2023-01-25T06:39:32.201 INFO:teuthology.orchestra.run.smithi117.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-137-generic 2023-01-25T06:39:32.352 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-modules-extra-5.4.0-137-generic (5.4.0-137.154) ... 2023-01-25T06:39:33.537 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-image-generic (5.4.0.137.135) ... 2023-01-25T06:39:33.663 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-generic (5.4.0.137.135) ... 2023-01-25T06:39:33.789 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-generic (5.4.0.137.135) ... 2023-01-25T06:39:33.924 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for linux-image-5.4.0-137-generic (5.4.0-137.154) ... 2023-01-25T06:39:34.002 INFO:teuthology.orchestra.run.smithi117.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-01-25T06:39:34.003 INFO:teuthology.orchestra.run.smithi117.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-137-generic 2023-01-25T06:39:56.911 INFO:teuthology.orchestra.run.smithi117.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-01-25T06:39:56.912 INFO:teuthology.orchestra.run.smithi117.stdout:Sourcing file `/etc/default/grub' 2023-01-25T06:39:56.929 INFO:teuthology.orchestra.run.smithi117.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-01-25T06:39:56.943 INFO:teuthology.orchestra.run.smithi117.stdout:Generating grub configuration file ... 2023-01-25T06:39:57.231 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.4.0-137-generic 2023-01-25T06:39:57.252 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.4.0-137-generic 2023-01-25T06:39:57.623 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-01-25T06:39:57.634 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-01-25T06:40:00.413 INFO:teuthology.orchestra.run.smithi117.stdout:done 2023-01-25T06:40:00.832 DEBUG:teuthology.orchestra.run.smithi117:> dpkg -s linux-image-generic 2023-01-25T06:40:00.879 INFO:teuthology.orchestra.run.smithi117.stdout:Package: linux-image-generic 2023-01-25T06:40:00.880 INFO:teuthology.orchestra.run.smithi117.stdout:Status: install ok installed 2023-01-25T06:40:00.880 INFO:teuthology.orchestra.run.smithi117.stdout:Priority: optional 2023-01-25T06:40:00.880 INFO:teuthology.orchestra.run.smithi117.stdout:Section: kernel 2023-01-25T06:40:00.880 INFO:teuthology.orchestra.run.smithi117.stdout:Installed-Size: 19 2023-01-25T06:40:00.881 INFO:teuthology.orchestra.run.smithi117.stdout:Maintainer: Ubuntu Kernel Team 2023-01-25T06:40:00.881 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: amd64 2023-01-25T06:40:00.881 INFO:teuthology.orchestra.run.smithi117.stdout:Source: linux-meta 2023-01-25T06:40:00.881 INFO:teuthology.orchestra.run.smithi117.stdout:Version: 5.4.0.137.135 2023-01-25T06:40:00.882 INFO:teuthology.orchestra.run.smithi117.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2023-01-25T06:40:00.882 INFO:teuthology.orchestra.run.smithi117.stdout:Depends: linux-image-5.4.0-137-generic, linux-modules-extra-5.4.0-137-generic, linux-firmware, intel-microcode, amd64-microcode 2023-01-25T06:40:00.882 INFO:teuthology.orchestra.run.smithi117.stdout:Recommends: thermald 2023-01-25T06:40:00.882 INFO:teuthology.orchestra.run.smithi117.stdout:Description: Generic Linux kernel image 2023-01-25T06:40:00.882 INFO:teuthology.orchestra.run.smithi117.stdout: This package will always depend on the latest generic kernel image 2023-01-25T06:40:00.883 INFO:teuthology.orchestra.run.smithi117.stdout: available. 2023-01-25T06:40:00.883 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-137-generic 2023-01-25T06:40:00.883 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T06:40:00.883 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T06:40:00.884 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-01-25T06:40:00.884 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T06:40:00.884 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get clean 2023-01-25T06:40:00.953 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2023-01-25T06:40:01.081 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-01-25T06:40:01.237 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-01-25T06:40:01.327 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-01-25T06:40:01.450 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-01-25T06:40:02.765 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2023-01-25T06:40:02.788 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-01-25T06:40:02.860 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2023-01-25T06:40:03.045 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2023-01-25T06:40:03.045 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2023-01-25T06:40:03.250 INFO:teuthology.orchestra.run.smithi117.stdout:linux-image-generic is already the newest version (5.4.0.137.135). 2023-01-25T06:40:03.251 INFO:teuthology.orchestra.run.smithi117.stdout:0 upgraded, 0 newly installed, 0 to remove and 129 not upgraded. 2023-01-25T06:40:03.253 DEBUG:teuthology.orchestra.run.smithi117:> dpkg -s linux-image-generic 2023-01-25T06:40:03.272 INFO:teuthology.orchestra.run.smithi117.stdout:Package: linux-image-generic 2023-01-25T06:40:03.273 INFO:teuthology.orchestra.run.smithi117.stdout:Status: install ok installed 2023-01-25T06:40:03.273 INFO:teuthology.orchestra.run.smithi117.stdout:Priority: optional 2023-01-25T06:40:03.273 INFO:teuthology.orchestra.run.smithi117.stdout:Section: kernel 2023-01-25T06:40:03.273 INFO:teuthology.orchestra.run.smithi117.stdout:Installed-Size: 19 2023-01-25T06:40:03.274 INFO:teuthology.orchestra.run.smithi117.stdout:Maintainer: Ubuntu Kernel Team 2023-01-25T06:40:03.274 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: amd64 2023-01-25T06:40:03.274 INFO:teuthology.orchestra.run.smithi117.stdout:Source: linux-meta 2023-01-25T06:40:03.274 INFO:teuthology.orchestra.run.smithi117.stdout:Version: 5.4.0.137.135 2023-01-25T06:40:03.274 INFO:teuthology.orchestra.run.smithi117.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2023-01-25T06:40:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:Depends: linux-image-5.4.0-137-generic, linux-modules-extra-5.4.0-137-generic, linux-firmware, intel-microcode, amd64-microcode 2023-01-25T06:40:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:Recommends: thermald 2023-01-25T06:40:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:Description: Generic Linux kernel image 2023-01-25T06:40:03.275 INFO:teuthology.orchestra.run.smithi117.stdout: This package will always depend on the latest generic kernel image 2023-01-25T06:40:03.275 INFO:teuthology.orchestra.run.smithi117.stdout: available. 2023-01-25T06:40:03.276 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2023-01-25T06:40:03.280 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.rqst61Jsmm 2023-01-25T06:40:03.281 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub/grub.cfg /tmp/tmp.rqst61Jsmm 2023-01-25T06:40:03.293 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.rqst61Jsmm 2023-01-25T06:40:03.340 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.rqst61Jsmm is 9KB 2023-01-25T06:40:03.392 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.rqst61Jsmm 2023-01-25T06:40:03.398 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-01-25T06:40:03.413 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:40:03.413 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-01-25T06:40:03.414 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-01-25T06:40:03.485 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-137-generic 2023-01-25T06:40:03.485 DEBUG:teuthology.orchestra.run.smithi117:> sudo update-grub 2023-01-25T06:40:04.770 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub' 2023-01-25T06:40:04.778 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-01-25T06:40:04.795 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2023-01-25T06:40:05.055 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.4.0-137-generic 2023-01-25T06:40:05.067 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.4.0-137-generic 2023-01-25T06:40:05.424 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-01-25T06:40:05.436 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-01-25T06:40:07.044 INFO:teuthology.orchestra.run.smithi117.stderr:done 2023-01-25T06:40:07.046 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2023-01-25T06:40:37.076 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-01-25T06:40:37.077 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-01-25T06:40:37.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:41:11.532 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2023-01-25T06:41:41.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-01-25T06:41:41.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:41:45.214 DEBUG:teuthology.orchestra.remote:No existing session 2023-01-25T06:42:15.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-01-25T06:42:15.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:42:15.293 DEBUG:teuthology.orchestra.run.smithi117:> true 2023-01-25T06:42:15.881 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-01-25T06:42:15.881 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-137-generic"... 2023-01-25T06:42:15.881 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2023-01-25T06:42:15.888 INFO:teuthology.orchestra.run.smithi117.stdout:5.4.0-137-generic 2023-01-25T06:42:15.888 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-137-generic vs 5.4.0-137-generic 2023-01-25T06:42:15.889 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T06:42:15.889 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-01-25T06:42:16.890 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T06:42:16.891 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T06:42:16.998 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2023-01-25T06:42:17.017 DEBUG:teuthology.parallel:result is None 2023-01-25T06:42:17.017 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T06:42:17.030 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T06:42:17.031 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T06:42:17.038 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T06:42:17.073 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T06:42:17.115 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T06:42:17.116 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T06:42:17.140 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T06:42:17.148 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T06:42:17.149 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T06:42:17.206 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T06:42:17.216 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T06:42:17.218 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T06:42:17.234 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T06:42:17.235 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T06:42:17.251 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T06:42:17.267 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T06:42:17.268 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T06:42:17.300 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T06:42:17.307 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T06:42:17.314 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:42:17.314 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T06:42:17.387 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2023-01-25T06:42:17.439 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T06:42:17.449 INFO:teuthology.task.internal:Starting timer... 2023-01-25T06:42:17.449 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T06:42:17.486 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T06:42:17.525 DEBUG:teuthology.task.selinux:Excluding smithi117: OS 'ubuntu' does not support SELinux 2023-01-25T06:42:17.526 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T06:42:17.526 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-01-25T06:42:17.526 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T06:42:17.527 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T06:42:17.567 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-25T06:42:17.568 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T06:42:17.585 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T06:42:17.587 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T06:46:58.680 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2023-01-25T06:46:58.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-01-25T06:46:58.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:46:58.731 DEBUG:teuthology.orchestra.run.smithi117:> true 2023-01-25T06:46:58.858 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T06:46:58.868 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T06:46:58.869 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T06:46:58.869 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T06:46:58.902 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-01-25T06:46:58.903 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: Command line: ntpd -gq 2023-01-25T06:46:58.904 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: proto: precision = 0.117 usec (-23) 2023-01-25T06:46:58.904 INFO:teuthology.orchestra.run.smithi117.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-01-25T06:46:58.905 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-01-25T06:46:58.905 INFO:teuthology.orchestra.run.smithi117.stderr:restrict ::: KOD does nothing without LIMITED. 2023-01-25T06:46:58.906 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: restrict ::: KOD does nothing without LIMITED. 2023-01-25T06:46:58.906 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: Listen and drop on 0 v6wildcard [::]:123 2023-01-25T06:46:58.907 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-01-25T06:46:58.907 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: Listen normally on 2 lo 127.0.0.1:123 2023-01-25T06:46:58.907 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: Listen normally on 3 enp3s0f1 172.21.15.117:123 2023-01-25T06:46:58.907 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: Listen normally on 4 lo [::1]:123 2023-01-25T06:46:58.908 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:4463%5]:123 2023-01-25T06:46:58.908 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:46:58 ntpd[19447]: Listening on routing socket on fd #22 for interface updates 2023-01-25T06:46:59.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:46:59.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:46:59.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:46:59.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:46:59.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:46:59.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:46:59.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:46:59.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:46:59.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:46:59.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:46:59.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:46:59.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:46:59 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:47:00.903 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:00 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:00.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:00 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:00.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:00 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:00.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:00 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:00.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:00 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:47:00.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:00 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:47:01.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:01.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:01.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:01.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:01.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:01.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:01.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:47:01.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:47:01.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:01.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:01.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:47:01.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:01 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:02 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:02 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:02 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:02.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:02 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:02.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:02 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:47:02.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:02 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:47:03.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:03.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:03.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:03.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:03.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:03.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:03.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:03.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:03.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:47:03.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:47:03.906 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:03.907 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:03.907 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:47:03.907 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:03 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:47:04.903 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:04 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:04 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:04 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:04 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:04 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:47:04.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:04 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:47:05.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:05.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:05.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:05.904 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:05.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-01-25T06:47:05.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: can't open /var/log/ntpstats/rawstats.20230125: Permission denied 2023-01-25T06:47:05.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-01-25T06:47:05.905 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: can't open /var/log/ntpstats/peerstats.20230125: Permission denied 2023-01-25T06:47:05.906 INFO:teuthology.orchestra.run.smithi117.stdout:25 Jan 06:47:05 ntpd[19447]: ntpd: time slew +0.007491 s 2023-01-25T06:47:05.906 INFO:teuthology.orchestra.run.smithi117.stdout:ntpd: time slew +0.007491s 2023-01-25T06:47:05.907 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-01-25T06:47:05.907 INFO:teuthology.orchestra.run.smithi117.stderr:25 Jan 06:47:05 ntpd[19447]: can't open /var/log/ntpstats/loopstats.20230125: Permission denied 2023-01-25T06:47:05.977 INFO:teuthology.orchestra.run.smithi117.stdout: remote refid st t when poll reach delay offset jitter 2023-01-25T06:47:05.977 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================== 2023-01-25T06:47:05.977 INFO:teuthology.orchestra.run.smithi117.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-01-25T06:47:05.978 INFO:teuthology.orchestra.run.smithi117.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-01-25T06:47:05.978 INFO:teuthology.orchestra.run.smithi117.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-01-25T06:47:05.978 INFO:teuthology.orchestra.run.smithi117.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-01-25T06:47:05.979 INFO:teuthology.run_tasks:Running task install... 2023-01-25T06:47:05.988 DEBUG:teuthology.task.install:project ceph 2023-01-25T06:47:05.988 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}} 2023-01-25T06:47:05.988 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'} 2023-01-25T06:47:05.989 INFO:teuthology.task.install:Using flavor: default 2023-01-25T06:47:05.995 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T06:47:05.995 INFO:teuthology.task.install:extra packages: [] 2023-01-25T06:47:05.996 DEBUG:teuthology.orchestra.run.smithi117:> sudo lsb_release -is 2023-01-25T06:47:06.043 INFO:teuthology.orchestra.run.smithi117.stdout:Ubuntu 2023-01-25T06:47:06.048 DEBUG:teuthology.misc:System to be installed: Ubuntu 2023-01-25T06:47:06.049 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-key list | grep Ceph 2023-01-25T06:47:06.147 INFO:teuthology.orchestra.run.smithi117.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-01-25T06:47:06.204 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-01-25T06:47:06.205 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph.com (release key) 2023-01-25T06:47:06.206 DEBUG:teuthology.orchestra.run.smithi117:> sudo lsb_release -is 2023-01-25T06:47:06.253 INFO:teuthology.orchestra.run.smithi117.stdout:Ubuntu 2023-01-25T06:47:06.258 DEBUG:teuthology.misc:System to be installed: Ubuntu 2023-01-25T06:47:06.258 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-01-25T06:47:06.259 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T06:47:06.414 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default/ 2023-01-25T06:47:06.414 INFO:teuthology.task.install.deb:Package version is 18.0.0-1959-gb0d25925-1focal 2023-01-25T06:47:06.528 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:47:06.528 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-01-25T06:47:06.545 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2023-01-25T06:47:06.684 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-01-25T06:47:06.691 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-01-25T06:47:06.708 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-01-25T06:47:06.744 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal InRelease 2023-01-25T06:47:06.768 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-01-25T06:47:06.793 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal Release.gpg 2023-01-25T06:47:06.816 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main i386 Packages [3,417 B] 2023-01-25T06:47:06.847 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-01-25T06:47:06.870 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-01-25T06:47:07.358 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 142 kB in 1s (219 kB/s) 2023-01-25T06:47:08.302 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2023-01-25T06:47:08.325 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-1959-gb0d25925-1focal cephadm=18.0.0-1959-gb0d25925-1focal ceph-mds=18.0.0-1959-gb0d25925-1focal ceph-mgr=18.0.0-1959-gb0d25925-1focal ceph-common=18.0.0-1959-gb0d25925-1focal ceph-fuse=18.0.0-1959-gb0d25925-1focal ceph-test=18.0.0-1959-gb0d25925-1focal ceph-volume=18.0.0-1959-gb0d25925-1focal radosgw=18.0.0-1959-gb0d25925-1focal python3-rados=18.0.0-1959-gb0d25925-1focal python3-rgw=18.0.0-1959-gb0d25925-1focal python3-cephfs=18.0.0-1959-gb0d25925-1focal python3-rbd=18.0.0-1959-gb0d25925-1focal libcephfs2=18.0.0-1959-gb0d25925-1focal libcephfs-dev=18.0.0-1959-gb0d25925-1focal librados2=18.0.0-1959-gb0d25925-1focal librbd1=18.0.0-1959-gb0d25925-1focal rbd-fuse=18.0.0-1959-gb0d25925-1focal 2023-01-25T06:47:08.393 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2023-01-25T06:47:08.570 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2023-01-25T06:47:08.571 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2023-01-25T06:47:08.725 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2023-01-25T06:47:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-01-25T06:47:08.726 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2023-01-25T06:47:08.726 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2023-01-25T06:47:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-01-25T06:47:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-01-25T06:47:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-01-25T06:47:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-01-25T06:47:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-01-25T06:47:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-01-25T06:47:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2023-01-25T06:47:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2023-01-25T06:47:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-01-25T06:47:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2023-01-25T06:47:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2023-01-25T06:47:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2023-01-25T06:47:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-01-25T06:47:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-01-25T06:47:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-01-25T06:47:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2023-01-25T06:47:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2023-01-25T06:47:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy python3-simplegeneric python3-simplejson 2023-01-25T06:47:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-01-25T06:47:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-01-25T06:47:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-01-25T06:47:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: qttranslations5-l10n socat xmlstarlet 2023-01-25T06:47:08.731 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2023-01-25T06:47:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-01-25T06:47:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-01-25T06:47:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-01-25T06:47:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-01-25T06:47:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-01-25T06:47:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-01-25T06:47:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-01-25T06:47:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-01-25T06:47:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-01-25T06:47:08.733 INFO:teuthology.orchestra.run.smithi117.stdout:Recommended packages: 2023-01-25T06:47:08.734 INFO:teuthology.orchestra.run.smithi117.stdout: btrfs-tools 2023-01-25T06:47:08.843 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2023-01-25T06:47:08.844 INFO:teuthology.orchestra.run.smithi117.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-01-25T06:47:08.844 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-01-25T06:47:08.844 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-01-25T06:47:08.844 INFO:teuthology.orchestra.run.smithi117.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2023-01-25T06:47:08.844 INFO:teuthology.orchestra.run.smithi117.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2023-01-25T06:47:08.845 INFO:teuthology.orchestra.run.smithi117.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2023-01-25T06:47:08.845 INFO:teuthology.orchestra.run.smithi117.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2023-01-25T06:47:08.846 INFO:teuthology.orchestra.run.smithi117.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2023-01-25T06:47:08.846 INFO:teuthology.orchestra.run.smithi117.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2023-01-25T06:47:08.846 INFO:teuthology.orchestra.run.smithi117.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2023-01-25T06:47:08.847 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-01-25T06:47:08.847 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2023-01-25T06:47:08.847 INFO:teuthology.orchestra.run.smithi117.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2023-01-25T06:47:08.847 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2023-01-25T06:47:08.847 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2023-01-25T06:47:08.848 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2023-01-25T06:47:08.848 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2023-01-25T06:47:08.848 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2023-01-25T06:47:08.848 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2023-01-25T06:47:08.848 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2023-01-25T06:47:08.849 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-01-25T06:47:08.849 INFO:teuthology.orchestra.run.smithi117.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2023-01-25T06:47:08.849 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2023-01-25T06:47:08.849 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse socat 2023-01-25T06:47:08.849 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet 2023-01-25T06:47:08.850 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages will be upgraded: 2023-01-25T06:47:08.850 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 librbd1 2023-01-25T06:47:08.906 INFO:teuthology.orchestra.run.smithi117.stdout:2 upgraded, 110 newly installed, 0 to remove and 124 not upgraded. 2023-01-25T06:47:08.906 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 210 MB of archives. 2023-01-25T06:47:08.906 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 1,029 MB of additional disk space will be used. 2023-01-25T06:47:08.907 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-01-25T06:47:08.988 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-01-25T06:47:08.991 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-01-25T06:47:09.003 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.0.0-1959-gb0d25925-1focal [3,427 kB] 2023-01-25T06:47:09.008 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-01-25T06:47:09.059 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-01-25T06:47:09.062 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-01-25T06:47:09.067 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-01-25T06:47:09.069 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-01-25T06:47:09.069 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-01-25T06:47:09.070 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-01-25T06:47:09.071 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-01-25T06:47:09.074 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-01-25T06:47:09.074 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-01-25T06:47:09.076 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-01-25T06:47:09.092 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-01-25T06:47:09.093 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-01-25T06:47:09.093 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-01-25T06:47:09.095 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-01-25T06:47:09.095 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-01-25T06:47:09.096 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-01-25T06:47:09.097 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-01-25T06:47:09.097 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-01-25T06:47:09.098 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-01-25T06:47:09.110 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-01-25T06:47:09.117 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-01-25T06:47:09.123 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-01-25T06:47:09.123 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-01-25T06:47:09.124 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-01-25T06:47:09.124 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-01-25T06:47:09.126 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2023-01-25T06:47:09.128 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-01-25T06:47:09.132 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-01-25T06:47:09.133 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2023-01-25T06:47:09.143 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-01-25T06:47:09.144 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-01-25T06:47:09.145 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-01-25T06:47:09.145 INFO:teuthology.orchestra.run.smithi117.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-01-25T06:47:09.146 INFO:teuthology.orchestra.run.smithi117.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-01-25T06:47:09.160 INFO:teuthology.orchestra.run.smithi117.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-01-25T06:47:09.162 INFO:teuthology.orchestra.run.smithi117.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-01-25T06:47:09.162 INFO:teuthology.orchestra.run.smithi117.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-01-25T06:47:09.163 INFO:teuthology.orchestra.run.smithi117.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-01-25T06:47:09.164 INFO:teuthology.orchestra.run.smithi117.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-01-25T06:47:09.177 INFO:teuthology.orchestra.run.smithi117.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-01-25T06:47:09.178 INFO:teuthology.orchestra.run.smithi117.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-01-25T06:47:09.179 INFO:teuthology.orchestra.run.smithi117.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-01-25T06:47:09.180 INFO:teuthology.orchestra.run.smithi117.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-01-25T06:47:09.181 INFO:teuthology.orchestra.run.smithi117.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-01-25T06:47:09.194 INFO:teuthology.orchestra.run.smithi117.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-01-25T06:47:09.195 INFO:teuthology.orchestra.run.smithi117.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-01-25T06:47:09.275 INFO:teuthology.orchestra.run.smithi117.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-01-25T06:47:09.357 INFO:teuthology.orchestra.run.smithi117.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-01-25T06:47:09.359 INFO:teuthology.orchestra.run.smithi117.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-01-25T06:47:09.377 INFO:teuthology.orchestra.run.smithi117.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-01-25T06:47:09.378 INFO:teuthology.orchestra.run.smithi117.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-01-25T06:47:09.378 INFO:teuthology.orchestra.run.smithi117.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-01-25T06:47:09.379 INFO:teuthology.orchestra.run.smithi117.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-01-25T06:47:09.379 INFO:teuthology.orchestra.run.smithi117.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-01-25T06:47:09.379 INFO:teuthology.orchestra.run.smithi117.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-01-25T06:47:09.380 INFO:teuthology.orchestra.run.smithi117.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-01-25T06:47:09.380 INFO:teuthology.orchestra.run.smithi117.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-01-25T06:47:09.381 INFO:teuthology.orchestra.run.smithi117.stdout:Get:63 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.0.0-1959-gb0d25925-1focal [3,961 kB] 2023-01-25T06:47:09.410 INFO:teuthology.orchestra.run.smithi117.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-01-25T06:47:09.411 INFO:teuthology.orchestra.run.smithi117.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-01-25T06:47:09.416 INFO:teuthology.orchestra.run.smithi117.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-01-25T06:47:09.419 INFO:teuthology.orchestra.run.smithi117.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-01-25T06:47:09.422 INFO:teuthology.orchestra.run.smithi117.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-01-25T06:47:09.423 INFO:teuthology.orchestra.run.smithi117.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-01-25T06:47:09.426 INFO:teuthology.orchestra.run.smithi117.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-01-25T06:47:09.429 INFO:teuthology.orchestra.run.smithi117.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-01-25T06:47:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-01-25T06:47:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2023-01-25T06:47:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-01-25T06:47:09.490 INFO:teuthology.orchestra.run.smithi117.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-01-25T06:47:09.490 INFO:teuthology.orchestra.run.smithi117.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-01-25T06:47:09.491 INFO:teuthology.orchestra.run.smithi117.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-01-25T06:47:09.491 INFO:teuthology.orchestra.run.smithi117.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-01-25T06:47:09.503 INFO:teuthology.orchestra.run.smithi117.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-01-25T06:47:09.503 INFO:teuthology.orchestra.run.smithi117.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-01-25T06:47:09.504 INFO:teuthology.orchestra.run.smithi117.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-01-25T06:47:09.504 INFO:teuthology.orchestra.run.smithi117.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-01-25T06:47:09.505 INFO:teuthology.orchestra.run.smithi117.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-01-25T06:47:09.545 INFO:teuthology.orchestra.run.smithi117.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.0.0-1959-gb0d25925-1focal [720 kB] 2023-01-25T06:47:09.568 INFO:teuthology.orchestra.run.smithi117.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.0.0-1959-gb0d25925-1focal [376 kB] 2023-01-25T06:47:09.586 INFO:teuthology.orchestra.run.smithi117.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.0.0-1959-gb0d25925-1focal [22.4 kB] 2023-01-25T06:47:09.587 INFO:teuthology.orchestra.run.smithi117.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.0.0-1959-gb0d25925-1focal [184 kB] 2023-01-25T06:47:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.0.0-1959-gb0d25925-1focal [46.5 kB] 2023-01-25T06:47:09.592 INFO:teuthology.orchestra.run.smithi117.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.0.0-1959-gb0d25925-1focal [354 kB] 2023-01-25T06:47:09.607 INFO:teuthology.orchestra.run.smithi117.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.0.0-1959-gb0d25925-1focal [6,511 kB] 2023-01-25T06:47:09.814 INFO:teuthology.orchestra.run.smithi117.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.0.0-1959-gb0d25925-1focal [104 kB] 2023-01-25T06:47:09.817 INFO:teuthology.orchestra.run.smithi117.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.0.0-1959-gb0d25925-1focal [466 kB] 2023-01-25T06:47:09.837 INFO:teuthology.orchestra.run.smithi117.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.0.0-1959-gb0d25925-1focal [29.6 MB] 2023-01-25T06:47:10.964 INFO:teuthology.orchestra.run.smithi117.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.0.0-1959-gb0d25925-1focal [10.7 MB] 2023-01-25T06:47:11.197 INFO:teuthology.orchestra.run.smithi117.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.0.0-1959-gb0d25925-1focal [221 kB] 2023-01-25T06:47:11.203 INFO:teuthology.orchestra.run.smithi117.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.0.0-1959-gb0d25925-1focal [133 kB] 2023-01-25T06:47:11.209 INFO:teuthology.orchestra.run.smithi117.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.0.0-1959-gb0d25925-1focal [1,534 kB] 2023-01-25T06:47:11.257 INFO:teuthology.orchestra.run.smithi117.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.0.0-1959-gb0d25925-1focal [7,861 kB] 2023-01-25T06:47:11.522 INFO:teuthology.orchestra.run.smithi117.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.0.0-1959-gb0d25925-1focal [28.9 MB] 2023-01-25T06:47:12.597 INFO:teuthology.orchestra.run.smithi117.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.0.0-1959-gb0d25925-1focal [3,808 B] 2023-01-25T06:47:12.598 INFO:teuthology.orchestra.run.smithi117.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.0.0-1959-gb0d25925-1focal [829 kB] 2023-01-25T06:47:12.610 INFO:teuthology.orchestra.run.smithi117.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.0.0-1959-gb0d25925-1focal [2,280 kB] 2023-01-25T06:47:12.626 INFO:teuthology.orchestra.run.smithi117.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.0.0-1959-gb0d25925-1focal [96.2 kB] 2023-01-25T06:47:12.627 INFO:teuthology.orchestra.run.smithi117.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.0.0-1959-gb0d25925-1focal [115 kB] 2023-01-25T06:47:12.628 INFO:teuthology.orchestra.run.smithi117.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.0.0-1959-gb0d25925-1focal [1,787 kB] 2023-01-25T06:47:12.658 INFO:teuthology.orchestra.run.smithi117.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.0.0-1959-gb0d25925-1focal [7,880 kB] 2023-01-25T06:47:12.912 INFO:teuthology.orchestra.run.smithi117.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.0.0-1959-gb0d25925-1focal [3,928 B] 2023-01-25T06:47:12.913 INFO:teuthology.orchestra.run.smithi117.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.0.0-1959-gb0d25925-1focal [53.4 MB] 2023-01-25T06:47:14.894 INFO:teuthology.orchestra.run.smithi117.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.0.0-1959-gb0d25925-1focal [225 kB] 2023-01-25T06:47:14.951 INFO:teuthology.orchestra.run.smithi117.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.0.0-1959-gb0d25925-1focal [27.4 kB] 2023-01-25T06:47:14.952 INFO:teuthology.orchestra.run.smithi117.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.0.0-1959-gb0d25925-1focal [13.3 MB] 2023-01-25T06:47:15.325 INFO:teuthology.orchestra.run.smithi117.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.0.0-1959-gb0d25925-1focal [97.2 kB] 2023-01-25T06:47:15.952 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 210 MB in 6s (32.5 MB/s) 2023-01-25T06:47:16.079 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-01-25T06:47:16.135 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141754 files and directories currently installed.) 2023-01-25T06:47:16.139 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-01-25T06:47:16.227 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-01-25T06:47:16.614 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-01-25T06:47:16.632 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-01-25T06:47:16.672 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-01-25T06:47:17.067 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-01-25T06:47:17.084 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-01-25T06:47:17.125 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-01-25T06:47:17.645 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-01-25T06:47:17.663 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-01-25T06:47:17.713 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-01-25T06:47:18.232 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-01-25T06:47:18.250 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-01-25T06:47:18.282 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-01-25T06:47:18.668 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-01-25T06:47:18.686 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-01-25T06:47:18.760 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-01-25T06:47:19.188 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-01-25T06:47:19.205 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-01-25T06:47:19.246 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-01-25T06:47:19.617 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../007-librbd1_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:19.732 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librbd1 (18.0.0-1959-gb0d25925-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2023-01-25T06:47:20.403 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../008-librados2_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:20.553 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librados2 (18.0.0-1959-gb0d25925-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2023-01-25T06:47:21.315 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs2. 2023-01-25T06:47:21.333 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../009-libcephfs2_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:21.399 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs2 (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:21.734 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rados. 2023-01-25T06:47:21.753 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../010-python3-rados_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:21.793 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rados (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:22.179 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-01-25T06:47:22.197 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.0.0-1959-gb0d25925-1focal_all.deb ... 2023-01-25T06:47:22.262 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-argparse (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:22.540 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cephfs. 2023-01-25T06:47:22.555 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../012-python3-cephfs_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:22.598 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cephfs (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:22.892 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-common. 2023-01-25T06:47:22.909 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../013-python3-ceph-common_18.0.0-1959-gb0d25925-1focal_all.deb ... 2023-01-25T06:47:22.942 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-common (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:23.262 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-prettytable. 2023-01-25T06:47:23.280 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2023-01-25T06:47:23.311 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-01-25T06:47:23.598 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rbd. 2023-01-25T06:47:23.616 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../015-python3-rbd_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:23.689 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rbd (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:24.050 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-01-25T06:47:24.069 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-01-25T06:47:24.109 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-01-25T06:47:24.521 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-01-25T06:47:24.540 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2023-01-25T06:47:24.584 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-01-25T06:47:24.937 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-01-25T06:47:24.956 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-01-25T06:47:24.995 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-01-25T06:47:25.356 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librgw2. 2023-01-25T06:47:25.374 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../019-librgw2_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:25.423 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librgw2 (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:26.308 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rgw. 2023-01-25T06:47:26.326 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../020-python3-rgw_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:26.392 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rgw (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:26.744 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liboath0. 2023-01-25T06:47:26.762 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2023-01-25T06:47:26.802 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-01-25T06:47:27.148 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libradosstriper1. 2023-01-25T06:47:27.168 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../022-libradosstriper1_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:27.231 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libradosstriper1 (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:27.976 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-common. 2023-01-25T06:47:27.994 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../023-ceph-common_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:28.035 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-common (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:31.039 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-base. 2023-01-25T06:47:31.057 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../024-ceph-base_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:31.108 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-base (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:32.552 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cherrypy3. 2023-01-25T06:47:32.568 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2023-01-25T06:47:32.644 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-01-25T06:47:33.022 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-natsort. 2023-01-25T06:47:33.038 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2023-01-25T06:47:33.080 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-01-25T06:47:33.425 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-logutils. 2023-01-25T06:47:33.442 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2023-01-25T06:47:33.509 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-01-25T06:47:33.844 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-markupsafe. 2023-01-25T06:47:33.860 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-01-25T06:47:33.902 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-01-25T06:47:34.197 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-mako. 2023-01-25T06:47:34.216 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-01-25T06:47:34.255 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-01-25T06:47:34.608 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplegeneric. 2023-01-25T06:47:34.625 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2023-01-25T06:47:34.675 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-01-25T06:47:34.986 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-singledispatch. 2023-01-25T06:47:35.004 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-01-25T06:47:35.044 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-01-25T06:47:35.363 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webob. 2023-01-25T06:47:35.383 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2_all.deb ... 2023-01-25T06:47:35.422 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-01-25T06:47:35.800 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-soupsieve. 2023-01-25T06:47:35.819 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-01-25T06:47:35.858 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-01-25T06:47:36.169 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-bs4. 2023-01-25T06:47:36.186 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2023-01-25T06:47:36.227 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-01-25T06:47:36.591 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-waitress. 2023-01-25T06:47:36.607 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-01-25T06:47:36.653 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-01-25T06:47:37.028 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-tempita. 2023-01-25T06:47:37.044 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2023-01-25T06:47:37.086 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-01-25T06:47:37.397 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-paste. 2023-01-25T06:47:37.417 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-01-25T06:47:37.455 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-01-25T06:47:37.808 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-01-25T06:47:37.828 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-01-25T06:47:37.909 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-01-25T06:47:38.236 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastedeploy. 2023-01-25T06:47:38.254 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2023-01-25T06:47:38.303 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-01-25T06:47:38.615 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webtest. 2023-01-25T06:47:38.631 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2023-01-25T06:47:38.673 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-01-25T06:47:39.026 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pecan. 2023-01-25T06:47:39.042 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-01-25T06:47:39.084 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-01-25T06:47:39.464 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-werkzeug. 2023-01-25T06:47:39.484 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2023-01-25T06:47:39.522 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2023-01-25T06:47:39.883 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-01-25T06:47:39.901 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.0.0-1959-gb0d25925-1focal_all.deb ... 2023-01-25T06:47:39.941 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-modules-core (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:40.346 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-01-25T06:47:40.362 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:40.404 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:40.757 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cryptography. 2023-01-25T06:47:40.777 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../045-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-01-25T06:47:40.815 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-01-25T06:47:41.177 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-openssl. 2023-01-25T06:47:41.193 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../046-python3-openssl_19.0.0-1build1_all.deb ... 2023-01-25T06:47:41.227 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-01-25T06:47:41.638 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr. 2023-01-25T06:47:41.655 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../047-ceph-mgr_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:41.696 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:42.233 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mon. 2023-01-25T06:47:42.250 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../048-ceph-mon_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:42.291 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mon (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:44.169 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-osd. 2023-01-25T06:47:44.186 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../049-ceph-osd_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:44.227 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-osd (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:47.014 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph. 2023-01-25T06:47:47.030 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../050-ceph_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:47.088 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:47.433 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-fuse. 2023-01-25T06:47:47.450 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../051-ceph-fuse_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:47.491 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-fuse (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:47.986 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mds. 2023-01-25T06:47:48.002 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../052-ceph-mds_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:48.044 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mds (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:48.589 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephadm. 2023-01-25T06:47:48.606 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../053-cephadm_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:47:48.647 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephadm (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:49.033 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-asyncssh. 2023-01-25T06:47:49.052 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../054-python3-asyncssh_1.12.2-1_all.deb ... 2023-01-25T06:47:49.116 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2023-01-25T06:47:49.511 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jinja2. 2023-01-25T06:47:49.529 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../055-python3-jinja2_2.10.1-2_all.deb ... 2023-01-25T06:47:49.569 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-01-25T06:47:49.922 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-01-25T06:47:49.938 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_18.0.0-1959-gb0d25925-1focal_all.deb ... 2023-01-25T06:47:50.005 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-cephadm (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:50.375 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-bcrypt. 2023-01-25T06:47:50.393 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../057-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-01-25T06:47:50.433 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-01-25T06:47:50.744 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jwt. 2023-01-25T06:47:50.760 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-01-25T06:47:50.803 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-01-25T06:47:51.156 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-attr. 2023-01-25T06:47:51.176 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2023-01-25T06:47:51.214 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-01-25T06:47:51.567 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pluggy. 2023-01-25T06:47:51.583 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2023-01-25T06:47:51.625 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-01-25T06:47:51.961 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-py. 2023-01-25T06:47:51.978 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-01-25T06:47:52.053 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-01-25T06:47:52.398 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-atomicwrites. 2023-01-25T06:47:52.414 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-01-25T06:47:52.455 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-01-25T06:47:52.783 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-packaging. 2023-01-25T06:47:52.799 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2023-01-25T06:47:52.841 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-packaging (20.3-1) ... 2023-01-25T06:47:53.211 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-wcwidth. 2023-01-25T06:47:53.229 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-01-25T06:47:53.269 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-01-25T06:47:53.589 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pytest. 2023-01-25T06:47:53.606 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2023-01-25T06:47:53.647 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-01-25T06:47:54.008 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-repoze.lru. 2023-01-25T06:47:54.028 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2023-01-25T06:47:54.100 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-01-25T06:47:54.386 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjs-underscore. 2023-01-25T06:47:54.405 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-01-25T06:47:54.452 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-01-25T06:47:54.746 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-01-25T06:47:54.763 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-01-25T06:47:54.804 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-01-25T06:47:55.132 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-routes. 2023-01-25T06:47:55.148 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2023-01-25T06:47:55.216 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-01-25T06:47:55.560 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-01-25T06:47:55.575 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.0.0-1959-gb0d25925-1focal_all.deb ... 2023-01-25T06:47:55.618 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-dashboard (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:47:56.157 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-decorator. 2023-01-25T06:47:56.174 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-01-25T06:47:56.215 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-01-25T06:47:56.535 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-01-25T06:47:56.551 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-01-25T06:47:56.593 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-01-25T06:47:56.967 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-scipy. 2023-01-25T06:47:56.983 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-01-25T06:47:57.016 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-01-25T06:47:58.342 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-01-25T06:47:58.359 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-01-25T06:47:58.408 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-01-25T06:47:59.084 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-joblib. 2023-01-25T06:47:59.104 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2023-01-25T06:47:59.472 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-01-25T06:47:59.853 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn. 2023-01-25T06:47:59.868 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-01-25T06:47:59.909 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-01-25T06:48:00.677 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-01-25T06:48:00.695 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.0.0-1959-gb0d25925-1focal_all.deb ... 2023-01-25T06:48:00.735 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:01.880 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cachetools. 2023-01-25T06:48:01.896 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2023-01-25T06:48:01.938 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-01-25T06:48:02.274 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1. 2023-01-25T06:48:02.291 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-01-25T06:48:02.332 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-01-25T06:48:02.718 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-01-25T06:48:02.737 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-01-25T06:48:02.776 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-01-25T06:48:03.207 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rsa. 2023-01-25T06:48:03.225 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2023-01-25T06:48:03.290 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rsa (4.0-3) ... 2023-01-25T06:48:03.627 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-google-auth. 2023-01-25T06:48:03.645 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2023-01-25T06:48:03.685 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-01-25T06:48:04.031 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-blinker. 2023-01-25T06:48:04.047 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-01-25T06:48:04.089 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-01-25T06:48:04.400 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-oauthlib. 2023-01-25T06:48:04.416 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-01-25T06:48:04.466 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-01-25T06:48:04.795 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-01-25T06:48:04.814 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-01-25T06:48:04.845 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-01-25T06:48:05.173 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-websocket. 2023-01-25T06:48:05.190 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-01-25T06:48:05.231 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-01-25T06:48:05.606 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-kubernetes. 2023-01-25T06:48:05.623 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-01-25T06:48:05.664 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-01-25T06:48:06.257 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-01-25T06:48:06.274 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.0.0-1959-gb0d25925-1focal_all.deb ... 2023-01-25T06:48:06.316 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:06.665 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libonig5:amd64. 2023-01-25T06:48:06.682 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2023-01-25T06:48:06.723 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-01-25T06:48:07.134 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjq1:amd64. 2023-01-25T06:48:07.152 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-01-25T06:48:07.192 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-01-25T06:48:07.446 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package jq. 2023-01-25T06:48:07.462 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-01-25T06:48:07.495 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-01-25T06:48:07.756 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package socat. 2023-01-25T06:48:07.772 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2023-01-25T06:48:07.814 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking socat (1.7.3.3-2) ... 2023-01-25T06:48:08.143 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package xmlstarlet. 2023-01-25T06:48:08.161 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-01-25T06:48:08.200 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-01-25T06:48:08.564 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-test. 2023-01-25T06:48:08.582 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../094-ceph-test_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:48:08.622 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-test (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:13.434 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-volume. 2023-01-25T06:48:13.450 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../095-ceph-volume_18.0.0-1959-gb0d25925-1focal_all.deb ... 2023-01-25T06:48:13.491 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-volume (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:13.821 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package fonts-mathjax. 2023-01-25T06:48:13.839 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-01-25T06:48:13.904 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-01-25T06:48:14.483 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs-dev. 2023-01-25T06:48:14.499 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../097-libcephfs-dev_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:48:14.540 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs-dev (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:14.944 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package nvme-cli. 2023-01-25T06:48:14.964 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-01-25T06:48:15.042 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-01-25T06:48:15.416 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-01-25T06:48:15.434 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2023-01-25T06:48:15.474 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2023-01-25T06:48:15.852 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjs-mathjax. 2023-01-25T06:48:15.872 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-01-25T06:48:15.943 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-01-25T06:48:17.627 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-natsort-doc. 2023-01-25T06:48:17.645 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2023-01-25T06:48:17.709 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-01-25T06:48:18.080 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webencodings. 2023-01-25T06:48:18.096 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-01-25T06:48:18.138 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-01-25T06:48:18.449 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-html5lib. 2023-01-25T06:48:18.469 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2023-01-25T06:48:18.507 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-01-25T06:48:18.918 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-01-25T06:48:18.936 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-01-25T06:48:18.976 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-01-25T06:48:19.513 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastescript. 2023-01-25T06:48:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2023-01-25T06:48:19.571 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-01-25T06:48:19.933 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-psutil. 2023-01-25T06:48:19.951 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-01-25T06:48:19.997 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-01-25T06:48:20.377 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyinotify. 2023-01-25T06:48:20.397 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-01-25T06:48:20.435 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-01-25T06:48:20.779 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplejson. 2023-01-25T06:48:20.799 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-01-25T06:48:20.838 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-01-25T06:48:21.140 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-01-25T06:48:21.157 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-01-25T06:48:21.198 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-01-25T06:48:21.927 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package radosgw. 2023-01-25T06:48:21.944 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../110-radosgw_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:48:21.985 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking radosgw (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:23.388 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package rbd-fuse. 2023-01-25T06:48:23.408 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../111-rbd-fuse_18.0.0-1959-gb0d25925-1focal_amd64.deb ... 2023-01-25T06:48:23.446 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking rbd-fuse (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:23.767 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-attr (19.3.0-2) ... 2023-01-25T06:48:24.084 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-01-25T06:48:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-01-25T06:48:24.361 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-01-25T06:48:24.579 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephadm (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:25.208 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user cephadm....done 2023-01-25T06:48:25.402 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-01-25T06:48:25.528 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-01-25T06:48:25.913 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-01-25T06:48:26.206 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-01-25T06:48:26.307 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-01-25T06:48:26.408 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-01-25T06:48:26.768 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-01-25T06:48:27.161 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-01-25T06:48:27.462 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-01-25T06:48:27.763 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-01-25T06:48:28.064 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-01-25T06:48:28.382 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-argparse (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-01-25T06:48:28.859 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-01-25T06:48:29.143 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-01-25T06:48:29.436 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-01-25T06:48:29.562 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-01-25T06:48:30.007 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-01-25T06:48:30.666 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-01-25T06:48:30.967 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-01-25T06:48:31.318 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-01-25T06:48:31.611 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-01-25T06:48:31.954 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-01-25T06:48:32.255 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-packaging (20.3-1) ... 2023-01-25T06:48:32.548 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-01-25T06:48:32.698 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-01-25T06:48:32.808 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-01-25T06:48:33.109 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-01-25T06:48:33.544 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2023-01-25T06:48:33.928 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-01-25T06:48:34.297 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up socat (1.7.3.3-2) ... 2023-01-25T06:48:34.448 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-common (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:34.766 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-01-25T06:48:34.900 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-01-25T06:48:35.293 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-01-25T06:48:35.419 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-01-25T06:48:35.587 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-01-25T06:48:35.879 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-01-25T06:48:36.198 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-01-25T06:48:36.324 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-01-25T06:48:36.466 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-01-25T06:48:36.801 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-01-25T06:48:37.094 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-01-25T06:48:37.403 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-01-25T06:48:37.551 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-01-25T06:48:37.871 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-01-25T06:48:38.173 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-01-25T06:48:38.482 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-01-25T06:48:38.830 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-01-25T06:48:39.101 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-01-25T06:48:39.227 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-01-25T06:48:39.587 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-01-25T06:48:39.946 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-01-25T06:48:40.072 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-01-25T06:48:40.215 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-01-25T06:48:40.558 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-01-25T06:48:40.875 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-01-25T06:48:42.339 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-01-25T06:48:42.482 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-01-25T06:48:42.850 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-01-25T06:48:43.159 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-01-25T06:48:43.460 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-01-25T06:48:43.620 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-01-25T06:48:43.963 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-01-25T06:48:44.281 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2023-01-25T06:48:44.674 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-01-25T06:48:45.059 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-01-25T06:48:46.094 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-01-25T06:48:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-01-25T06:48:46.363 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-01-25T06:48:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rsa (4.0-3) ... 2023-01-25T06:48:46.816 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-01-25T06:48:46.941 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-01-25T06:48:47.251 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-01-25T06:48:47.569 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-01-25T06:48:47.929 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-01-25T06:48:48.289 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librados2 (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:48.423 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2023-01-25T06:48:48.582 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librgw2 (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:48.708 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libsqlite3-mod-ceph (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:48.945 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs2 (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:49.096 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-routes (2.4.1-2) ... 2023-01-25T06:48:49.398 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libradosstriper1 (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:49.523 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-01-25T06:48:49.834 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librbd1 (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:49.978 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-modules-core (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:50.115 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-fuse (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:50.338 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-01-25T06:48:50.338 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-01-25T06:48:50.833 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs-dev (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:50.959 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rados (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:51.118 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-01-25T06:48:52.037 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rbd (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:52.155 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up rbd-fuse (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:52.323 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rgw (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:52.449 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cephfs (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:52.600 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-common (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:53.195 INFO:teuthology.orchestra.run.smithi117.stdout:Adding group ceph....done 2023-01-25T06:48:53.605 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user ceph....done 2023-01-25T06:48:53.806 INFO:teuthology.orchestra.run.smithi117.stdout:Setting system user ceph properties....done 2023-01-25T06:48:53.824 INFO:teuthology.orchestra.run.smithi117.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-01-25T06:48:53.975 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-01-25T06:48:54.277 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-01-25T06:48:54.760 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-test (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:54.886 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up radosgw (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:55.130 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-01-25T06:48:55.131 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-01-25T06:48:55.805 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-base (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:56.312 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-01-25T06:48:56.766 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mds (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:57.027 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-01-25T06:48:57.027 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-01-25T06:48:57.475 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:57.710 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-01-25T06:48:57.710 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-01-25T06:48:58.160 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-osd (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:58.682 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-01-25T06:48:58.683 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-01-25T06:48:59.120 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-k8sevents (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:59.280 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:59.481 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mon (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:48:59.733 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-01-25T06:48:59.733 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-01-25T06:49:00.200 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-cephadm (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:49:00.342 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:49:00.485 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-dashboard (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:49:00.628 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-volume (18.0.0-1959-gb0d25925-1focal) ... 2023-01-25T06:49:00.780 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-01-25T06:49:00.931 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-01-25T06:49:01.550 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-01-25T06:49:04.468 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-01-25T06:49:05.095 INFO:teuthology.orchestra.run.smithi117.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-01-25T06:49:05.097 DEBUG:teuthology.parallel:result is None 2023-01-25T06:49:05.098 DEBUG:teuthology.orchestra.run.smithi117:> sudo lsb_release -is 2023-01-25T06:49:05.142 INFO:teuthology.orchestra.run.smithi117.stdout:Ubuntu 2023-01-25T06:49:05.146 DEBUG:teuthology.misc:System to be installed: Ubuntu 2023-01-25T06:49:05.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T06:49:05.294 DEBUG:teuthology.orchestra.run.smithi117:> dpkg-query -W -f '${Version}' ceph 2023-01-25T06:49:05.317 INFO:teuthology.orchestra.run.smithi117.stdout:18.0.0-1959-gb0d25925-1focal 2023-01-25T06:49:05.317 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959-gb0d25925-1focal 2023-01-25T06:49:05.318 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959-gb0d25925-1focal is installed. 2023-01-25T06:49:05.318 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T06:49:05.319 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:05.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T06:49:05.375 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T06:49:05.375 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:05.376 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T06:49:05.387 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T06:49:05.398 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T06:49:05.398 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:05.399 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T06:49:05.454 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T06:49:05.470 INFO:teuthology.run_tasks:Running task ceph... 2023-01-25T06:49:05.548 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-25T06:49:05.549 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /var/log/ceph 2023-01-25T06:49:05.565 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-25T06:49:05.565 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T06:49:05.623 INFO:tasks.ceph:Creating extra log directories... 2023-01-25T06:49:05.624 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T06:49:05.643 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-25T06:49:05.644 INFO:tasks.ceph:config {'conf': {'global': {'debug client': 20, 'debug mds': 20, 'debug ms': 1, 'osd max object name len': 460, 'osd max object namespace len': 64, 'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-25T06:49:05.644 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136940', 'branch': 'wip-yuri-testing-2023-01-23-0926', 'description': 'rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7136940', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-01-23-0926'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'workunit': {'branch': 'wip-yuri-testing-2023-01-23-0926', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1']], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri-testing-2023-01-23-0926', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'targets': {'smithi117.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'global': {'debug client': 20, 'debug mds': 20, 'debug ms': 1, 'osd max object name len': 460, 'osd max object namespace len': 64, 'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'log-ignorelist': ['overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'cluster': 'ceph'}}, {'exec': {'client.0': ['ceph osd pool create data_cache 4', 'ceph osd tier add cephfs_data data_cache', 'ceph osd tier cache-mode data_cache writeback', 'ceph osd tier set-overlay cephfs_data data_cache', 'ceph osd pool set data_cache hit_set_type bloom', 'ceph osd pool set data_cache hit_set_count 8', 'ceph osd pool set data_cache hit_set_period 3600', 'ceph osd pool set data_cache min_read_recency_for_promote 0']}}, {'ceph-fuse': None}, {'exec': {'client.0': ['sudo chmod 777 $TESTDIR/mnt.0/', 'dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5', 'ls -al $TESTDIR/mnt.0/foo', 'truncate --size 0 $TESTDIR/mnt.0/foo', 'ls -al $TESTDIR/mnt.0/foo', 'dd if=/dev/urandom of=$TESTDIR/mnt.0/foo bs=1M count=5', 'ls -al $TESTDIR/mnt.0/foo', 'cp $TESTDIR/mnt.0/foo /tmp/foo', 'sync', 'rados -p data_cache ls -', 'sleep 10', 'rados -p data_cache ls -', 'rados -p data_cache cache-flush-evict-all', 'rados -p data_cache ls -', 'sleep 1']}}, {'exec': {'client.1': ['hexdump -C /tmp/foo | head', 'hexdump -C $TESTDIR/mnt.1/foo | head', 'cmp $TESTDIR/mnt.1/foo /tmp/foo']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_22:20:59', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-25T06:49:05.645 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T06:49:05.651 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T06:49:05.711 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:05.711 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T06:49:05.716 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T06:49:05.717 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2023-01-25T06:49:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T06:49:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:49:05.723 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 359 Links: 1 2023-01-25T06:49:05.724 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:49:05.724 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-01-25 06:45:41.768422546 +0000 2023-01-25T06:49:05.724 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-01-25 06:45:41.484484474 +0000 2023-01-25T06:49:05.725 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-01-25 06:45:41.484484474 +0000 2023-01-25T06:49:05.725 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2023-01-25T06:49:05.726 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T06:49:05.740 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-01-25T06:49:05.741 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-01-25T06:49:05.741 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000398618 s, 1.3 MB/s 2023-01-25T06:49:05.742 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T06:49:05.791 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2023-01-25T06:49:05.798 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T06:49:05.799 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:49:05.799 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 363 Links: 1 2023-01-25T06:49:05.799 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:49:05.799 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-01-25 06:45:41.988374576 +0000 2023-01-25T06:49:05.800 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-01-25 06:45:41.984375448 +0000 2023-01-25T06:49:05.800 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-01-25 06:45:41.984375448 +0000 2023-01-25T06:49:05.800 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2023-01-25T06:49:05.801 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T06:49:05.814 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-01-25T06:49:05.815 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-01-25T06:49:05.815 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000366282 s, 1.4 MB/s 2023-01-25T06:49:05.816 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T06:49:05.868 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2023-01-25T06:49:05.875 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T06:49:05.875 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:49:05.875 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 372 Links: 1 2023-01-25T06:49:05.876 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:49:05.876 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-01-25 06:45:42.724218435 +0000 2023-01-25T06:49:05.876 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-01-25 06:45:42.460274416 +0000 2023-01-25T06:49:05.876 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-01-25 06:45:42.460274416 +0000 2023-01-25T06:49:05.877 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2023-01-25T06:49:05.877 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T06:49:05.935 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-01-25T06:49:05.935 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-01-25T06:49:05.936 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000396175 s, 1.3 MB/s 2023-01-25T06:49:05.937 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T06:49:05.945 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2023-01-25T06:49:05.952 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T06:49:05.952 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:49:05.953 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 377 Links: 1 2023-01-25T06:49:05.953 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:49:05.953 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-01-25 06:45:42.960168391 +0000 2023-01-25T06:49:05.953 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-01-25 06:45:42.956169239 +0000 2023-01-25T06:49:05.953 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-01-25 06:45:42.956169239 +0000 2023-01-25T06:49:05.954 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2023-01-25T06:49:05.954 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T06:49:05.969 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-01-25T06:49:05.969 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-01-25T06:49:05.969 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000383644 s, 1.3 MB/s 2023-01-25T06:49:05.971 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T06:49:05.978 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-25T06:49:05.979 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-01-25T06:49:05.979 INFO:tasks.ceph:Generating config... 2023-01-25T06:49:05.980 INFO:tasks.ceph:[global] debug client = 20 2023-01-25T06:49:05.980 INFO:tasks.ceph:[global] debug mds = 20 2023-01-25T06:49:05.981 INFO:tasks.ceph:[global] debug ms = 1 2023-01-25T06:49:05.981 INFO:tasks.ceph:[global] osd max object name len = 460 2023-01-25T06:49:05.981 INFO:tasks.ceph:[global] osd max object namespace len = 64 2023-01-25T06:49:05.981 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-01-25T06:49:05.982 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-25T06:49:05.982 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-25T06:49:05.982 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-25T06:49:05.983 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-25T06:49:05.983 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-25T06:49:05.983 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-25T06:49:05.983 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-25T06:49:05.984 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-25T06:49:05.984 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-25T06:49:05.984 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-25T06:49:05.985 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-25T06:49:05.985 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-25T06:49:05.985 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-25T06:49:05.985 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-25T06:49:05.986 INFO:tasks.ceph:Setting up mon.a... 2023-01-25T06:49:05.986 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-25T06:49:06.058 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.keyring 2023-01-25T06:49:06.072 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-25T06:49:06.115 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T06:49:06.126 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.117')] 2023-01-25T06:49:06.127 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.117', 'debug client': 20, 'debug mds': 20, 'debug ms': 1, 'osd max object name len': 460, 'osd max object namespace len': 64, 'mon election default strategy': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mds.a': {}} 2023-01-25T06:49:06.128 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:06.129 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T06:49:06.173 DEBUG:teuthology.orchestra.run.smithi117:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.117 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:49:06.211 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:49:06.212 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool: generated fsid 52037823-4da1-437a-aebe-937b909d2d65 2023-01-25T06:49:06.212 INFO:teuthology.orchestra.run.smithi117.stdout:setting min_mon_release = pacific 2023-01-25T06:49:06.212 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 0 2023-01-25T06:49:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:fsid 52037823-4da1-437a-aebe-937b909d2d65 2023-01-25T06:49:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:last_changed 2023-01-25T06:49:06.212678+0000 2023-01-25T06:49:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:created 2023-01-25T06:49:06.212678+0000 2023-01-25T06:49:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:min_mon_release 16 (pacific) 2023-01-25T06:49:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:election_strategy: 3 2023-01-25T06:49:06.214 INFO:teuthology.orchestra.run.smithi117.stdout:0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2023-01-25T06:49:06.214 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-01-25T06:49:06.215 DEBUG:teuthology.orchestra.run.smithi117:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T06:49:06.220 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 52037823-4da1-437a-aebe-937b909d2d65... 2023-01-25T06:49:06.221 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T06:49:06.296 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2023-01-25T06:49:06.296 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2023-01-25T06:49:06.297 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T06:49:06.297 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2023-01-25T06:49:06.297 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.297 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2023-01-25T06:49:06.298 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.298 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2023-01-25T06:49:06.298 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.298 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2023-01-25T06:49:06.298 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2023-01-25T06:49:06.299 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.299 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2023-01-25T06:49:06.299 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2023-01-25T06:49:06.300 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.300 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T06:49:06.300 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2023-01-25T06:49:06.300 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.301 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T06:49:06.301 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T06:49:06.301 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.301 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2023-01-25T06:49:06.301 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.302 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2023-01-25T06:49:06.302 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2023-01-25T06:49:06.302 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2023-01-25T06:49:06.302 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2023-01-25T06:49:06.302 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2023-01-25T06:49:06.303 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2023-01-25T06:49:06.303 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2023-01-25T06:49:06.303 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T06:49:06.303 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T06:49:06.303 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2023-01-25T06:49:06.304 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.304 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T06:49:06.304 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.304 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2023-01-25T06:49:06.305 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.305 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2023-01-25T06:49:06.305 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.306 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2023-01-25T06:49:06.306 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2023-01-25T06:49:06.306 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2023-01-25T06:49:06.307 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = 172.21.15.117 2023-01-25T06:49:06.307 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2023-01-25T06:49:06.307 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2023-01-25T06:49:06.308 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2023-01-25T06:49:06.308 INFO:teuthology.orchestra.run.smithi117.stdout: osd max object name len = 460 2023-01-25T06:49:06.309 INFO:teuthology.orchestra.run.smithi117.stdout: osd max object namespace len = 64 2023-01-25T06:49:06.309 INFO:teuthology.orchestra.run.smithi117.stdout: mon election default strategy = 3 2023-01-25T06:49:06.309 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = 52037823-4da1-437a-aebe-937b909d2d65 2023-01-25T06:49:06.310 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.310 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2023-01-25T06:49:06.310 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2023-01-25T06:49:06.311 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.311 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2023-01-25T06:49:06.312 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2023-01-25T06:49:06.312 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.312 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2023-01-25T06:49:06.313 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2023-01-25T06:49:06.313 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.313 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2023-01-25T06:49:06.313 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2023-01-25T06:49:06.313 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2023-01-25T06:49:06.314 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.314 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2023-01-25T06:49:06.314 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2023-01-25T06:49:06.314 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.314 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2023-01-25T06:49:06.315 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.315 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2023-01-25T06:49:06.315 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.315 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2023-01-25T06:49:06.315 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.316 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2023-01-25T06:49:06.316 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2023-01-25T06:49:06.316 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.316 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2023-01-25T06:49:06.316 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2023-01-25T06:49:06.317 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore zero block detection = True 2023-01-25T06:49:06.317 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2023-01-25T06:49:06.317 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2023-01-25T06:49:06.317 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify cached snaps = True 2023-01-25T06:49:06.317 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify missing on start = True 2023-01-25T06:49:06.318 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock override recovery settings = True 2023-01-25T06:49:06.318 INFO:teuthology.orchestra.run.smithi117.stdout: osd op queue = debug_random 2023-01-25T06:49:06.318 INFO:teuthology.orchestra.run.smithi117.stdout: osd op queue cut off = debug_random 2023-01-25T06:49:06.318 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.318 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2023-01-25T06:49:06.319 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2023-01-25T06:49:06.319 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2023-01-25T06:49:06.319 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2023-01-25T06:49:06.319 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2023-01-25T06:49:06.319 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2023-01-25T06:49:06.320 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2023-01-25T06:49:06.320 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2023-01-25T06:49:06.320 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.320 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2023-01-25T06:49:06.320 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2023-01-25T06:49:06.321 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2023-01-25T06:49:06.321 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2023-01-25T06:49:06.321 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2023-01-25T06:49:06.321 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2023-01-25T06:49:06.321 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2023-01-25T06:49:06.322 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2023-01-25T06:49:06.322 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2023-01-25T06:49:06.322 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2023-01-25T06:49:06.322 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2023-01-25T06:49:06.322 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.323 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T06:49:06.323 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T06:49:06.323 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T06:49:06.323 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.323 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T06:49:06.324 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T06:49:06.324 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T06:49:06.324 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.324 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2023-01-25T06:49:06.324 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2023-01-25T06:49:06.325 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.325 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2023-01-25T06:49:06.325 INFO:teuthology.orchestra.run.smithi117.stdout: mon scrub interval = 300 2023-01-25T06:49:06.325 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:06.325 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2023-01-25T06:49:06.326 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2023-01-25T06:49:06.326 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2023-01-25T06:49:06.326 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2023-01-25T06:49:06.326 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T06:49:06.327 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T06:49:06.327 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2023-01-25T06:49:06.327 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2023-01-25T06:49:06.328 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-25T06:49:06.329 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-25T06:49:06.383 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-25T06:49:06.384 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:06.384 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-25T06:49:06.392 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:06.392 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-25T06:49:06.441 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi117.front.sepia.ceph.com 2023-01-25T06:49:06.441 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:06.442 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T06:49:06.442 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T06:49:06.469 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:06.469 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:49:06.473 INFO:tasks.ceph:Setting up mon nodes... 2023-01-25T06:49:06.474 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-25T06:49:06.474 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T06:49:06.517 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T06:49:06.520 INFO:tasks.ceph:Setting up mds nodes... 2023-01-25T06:49:06.520 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-01-25T06:49:06.612 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-01-25T06:49:06.616 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-01-25T06:49:06.628 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-25T06:49:06.629 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-25T06:49:06.665 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-25T06:49:06.677 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-25T06:49:06.761 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-25T06:49:06.774 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-25T06:49:06.774 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-01-25T06:49:06.775 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-25T06:49:06.791 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-25T06:49:06.792 INFO:tasks.ceph:role: osd.0 2023-01-25T06:49:06.792 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com 2023-01-25T06:49:06.792 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T06:49:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:49:07.324 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:49:07.324 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:49:07.324 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2023-01-25T06:49:07.324 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:49:07.325 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:49:07.325 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:49:07.325 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:49:07.325 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:49:07.325 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:49:07.326 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2023-01-25T06:49:07.327 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-25T06:49:07.426 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-25T06:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2023-01-25T06:49:07.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T06:49:07.440 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-25T06:49:07.451 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-25T06:49:07.451 INFO:tasks.ceph:role: osd.1 2023-01-25T06:49:07.452 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com 2023-01-25T06:49:07.452 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T06:49:07.974 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:49:07.975 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:49:07.975 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:49:07.975 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2023-01-25T06:49:07.975 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:49:07.976 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:49:07.976 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:49:07.976 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:49:07.976 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:49:07.976 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:49:07.978 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2023-01-25T06:49:07.978 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-25T06:49:08.020 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-25T06:49:08.031 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2023-01-25T06:49:08.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T06:49:08.033 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-25T06:49:08.043 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-25T06:49:08.043 INFO:tasks.ceph:role: osd.2 2023-01-25T06:49:08.044 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com 2023-01-25T06:49:08.044 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T06:49:08.499 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:49:08.499 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:49:08.499 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:49:08.500 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 2023-01-25T06:49:08.500 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:49:08.500 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:49:08.500 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:49:08.501 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:49:08.501 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:49:08.501 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:49:08.502 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2023-01-25T06:49:08.502 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-25T06:49:08.561 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-25T06:49:08.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T06:49:08.573 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: /sbin/restorecon: command not found 2023-01-25T06:49:08.573 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:49:08.648 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:08.646+0000 7f620318d680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-25T06:49:08.651 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:08.650+0000 7f620318d680 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-25T06:49:08.652 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:08.650+0000 7f620318d680 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T06:49:08.652 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:08.650+0000 7f620318d680 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T06:49:08.652 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:08.650+0000 7f620318d680 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T06:49:08.653 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:08.650+0000 7f620318d680 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-25T06:49:09.223 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:09.222+0000 7f620318d680 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:49:09.998 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:09.994+0000 7f620318d680 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:49:10.862 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-25T06:49:10.879 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:10.970+0000 7fb2fbea3680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-25T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:10.970+0000 7fb2fbea3680 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-25T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:10.970+0000 7fb2fbea3680 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T06:49:10.975 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:10.970+0000 7fb2fbea3680 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:10.970+0000 7fb2fbea3680 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T06:49:10.976 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:10.974+0000 7fb2fbea3680 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-25T06:49:11.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:11.533+0000 7fb2fbea3680 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:49:12.321 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:12.317+0000 7fb2fbea3680 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:49:13.170 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-25T06:49:13.184 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:49:13.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:13.225+0000 7f974f4f8680 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-25T06:49:13.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:13.229+0000 7f974f4f8680 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-25T06:49:13.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:13.229+0000 7f974f4f8680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T06:49:13.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:13.229+0000 7f974f4f8680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T06:49:13.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:13.229+0000 7f974f4f8680 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T06:49:13.233 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:13.229+0000 7f974f4f8680 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-25T06:49:13.792 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:13.789+0000 7f974f4f8680 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:49:14.577 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:14.573+0000 7f974f4f8680 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:49:15.433 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-25T06:49:15.448 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-25T06:49:15.449 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:15.449 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-25T06:49:15.506 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:15.506 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-01-25T06:49:15.520 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:15.520 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-25T06:49:15.535 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:15.536 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-25T06:49:15.549 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:15.550 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-25T06:49:15.564 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:15.565 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-25T06:49:15.618 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:15.618 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-25T06:49:15.625 INFO:tasks.ceph:Adding keys to all mons... 2023-01-25T06:49:15.625 DEBUG:teuthology.orchestra.run.smithi117:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T06:49:15.636 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.x] 2023-01-25T06:49:15.636 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDi0NBjPxDoHhAAmzLDHilZczZ0cgMhcX73mg== 2023-01-25T06:49:15.636 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2023-01-25T06:49:15.636 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDi0NBjgISSJBAA/WrjaqdUe/M5vigjtE7CxQ== 2023-01-25T06:49:15.637 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.0] 2023-01-25T06:49:15.637 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDk0NBjcxezJhAAp+bzPTPctTMVUN/Zh2zaIg== 2023-01-25T06:49:15.637 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.1] 2023-01-25T06:49:15.637 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDm0NBjwHYmOhAAOEghn17Z1RmENj4F5F4qDA== 2023-01-25T06:49:15.638 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.2] 2023-01-25T06:49:15.638 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDp0NBj3NvXDRAAbzgGJO9v4XSUbMrQXolD8g== 2023-01-25T06:49:15.638 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2023-01-25T06:49:15.638 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDi0NBjwxaxJxAABCGKG+KhFyU9bRECheg8YA== 2023-01-25T06:49:15.638 INFO:teuthology.orchestra.run.smithi117.stdout:[client.1] 2023-01-25T06:49:15.639 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDi0NBjA0JsLRAA7ODtWxrYCoMAHbnpnT/2FA== 2023-01-25T06:49:15.640 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T06:49:15.686 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-01-25T06:49:15.735 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:49:15.786 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:49:15.831 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:49:15.926 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T06:49:15.976 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T06:49:16.022 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-25T06:49:16.023 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-25T06:49:16.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T06:49:16.149 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-25T06:49:16.163 DEBUG:teuthology.orchestra.run.smithi117:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:49:16.171 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-25T06:49:16.172 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-25T06:49:16.172 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-25T06:49:16.176 INFO:tasks.ceph.mon.a:Started 2023-01-25T06:49:16.177 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph config set mgr mgr_pool false --force 2023-01-25T06:49:16.279 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9c5668700 1 Processor -- start 2023-01-25T06:49:16.280 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9c5668700 1 -- start start 2023-01-25T06:49:16.281 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9c5668700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 0x7fb9c012ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:16.282 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9c5668700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c009ad90 con 0x7fb9c012f470 2023-01-25T06:49:16.282 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9c5668700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c006f590 con 0x7fb9c012eb60 2023-01-25T06:49:16.282 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9beffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 0x7fb9c012ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:16.282 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9beffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 0x7fb9c012ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33856/0 (socket says 172.21.15.117:33856) 2023-01-25T06:49:16.283 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9beffd700 1 -- 172.21.15.117:0/1425829447 learned_addr learned my addr 172.21.15.117:0/1425829447 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:16.283 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9be7fc700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb9c012f470 0x7fb9c006d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:44360/0 (socket says 172.21.15.117:44360) 2023-01-25T06:49:16.283 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9beffd700 1 -- 172.21.15.117:0/1425829447 >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 msgr2=0x7fb9c012ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T06:49:16.283 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9beffd700 1 -- 172.21.15.117:0/1425829447 >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 msgr2=0x7fb9c012ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T06:49:16.283 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9beffd700 1 --2- 172.21.15.117:0/1425829447 >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 0x7fb9c012ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T06:49:16.284 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.277+0000 7fb9beffd700 1 --2- 172.21.15.117:0/1425829447 >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 0x7fb9c012ef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T06:49:16.325 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753137377 0 0) 0x7fb9c009ad90 con 0x7fb9c012f470 2023-01-25T06:49:16.326 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9a80057e0 con 0x7fb9c012f470 2023-01-25T06:49:16.326 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 <== mon.0 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1510950071 0 0) 0x7fb9a80057e0 con 0x7fb9c012f470 2023-01-25T06:49:16.326 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 msgr2=0x7fb9c012ef30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-01-25T06:49:16.326 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 --2- 172.21.15.117:0/1425829447 >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 0x7fb9c012ef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 --> v1:172.21.15.117:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c009aa80 con 0x7fb9c012f470 2023-01-25T06:49:16.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 <== mon.0 v1:172.21.15.117:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 2547789375 0 0) 0x7fb9b00015a0 con 0x7fb9c012f470 2023-01-25T06:49:16.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 >> v1:172.21.15.117:6789/0 conn(0x7fb9c012f470 legacy=0x7fb9c006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:16.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 --2- 172.21.15.117:0/1425829447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9a80060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:16.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a8006770 con 0x7fb9a8005cb0 2023-01-25T06:49:16.328 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9beffd700 1 --2- 172.21.15.117:0/1425829447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9a80060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:16.328 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9beffd700 1 -- 172.21.15.117:0/1425829447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a80057e0 con 0x7fb9a8005cb0 2023-01-25T06:49:16.329 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9beffd700 1 --2- 172.21.15.117:0/1425829447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9a80060a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40098d0 tx=0x7fb9a8007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab32a6f8683086c server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:16.329 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9b400c070 con 0x7fb9a8005cb0 2023-01-25T06:49:16.330 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4004170 con 0x7fb9a8005cb0 2023-01-25T06:49:16.330 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.321+0000 7fb9bdffb700 1 -- 172.21.15.117:0/1425829447 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9b4004450 con 0x7fb9a8005cb0 2023-01-25T06:49:16.330 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 -- 172.21.15.117:0/1425829447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 msgr2=0x7fb9a80060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:16.330 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 --2- 172.21.15.117:0/1425829447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9a80060a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40098d0 tx=0x7fb9a8007ab0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 -- 172.21.15.117:0/1425829447 shutdown_connections 2023-01-25T06:49:16.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 --2- 172.21.15.117:0/1425829447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9a80060a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 --2- 172.21.15.117:0/1425829447 >> v2:172.21.15.117:3300/0 conn(0x7fb9c012eb60 0x7fb9c012ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 -- 172.21.15.117:0/1425829447 >> 172.21.15.117:0/1425829447 conn(0x7fb9c00a0db0 msgr2=0x7fb9c00a11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:16.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 -- 172.21.15.117:0/1425829447 shutdown_connections 2023-01-25T06:49:16.332 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 -- 172.21.15.117:0/1425829447 wait complete. 2023-01-25T06:49:16.334 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 Processor -- start 2023-01-25T06:49:16.334 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.325+0000 7fb9c5668700 1 -- start start 2023-01-25T06:49:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9c5668700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9c01c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9c5668700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c011c960 con 0x7fb9a8005cb0 2023-01-25T06:49:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9c01c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:16.335 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9c01c5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33870/0 (socket says 172.21.15.117:33870) 2023-01-25T06:49:16.336 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9beffd700 1 -- 172.21.15.117:0/4069810408 learned_addr learned my addr 172.21.15.117:0/4069810408 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:16.336 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9beffd700 1 -- 172.21.15.117:0/4069810408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0065720 con 0x7fb9a8005cb0 2023-01-25T06:49:16.336 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9beffd700 1 --2- 172.21.15.117:0/4069810408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9c01c5d00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4016040 tx=0x7fb9b40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:16.336 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9a77fe700 1 -- 172.21.15.117:0/4069810408 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9b4003fd0 con 0x7fb9a8005cb0 2023-01-25T06:49:16.337 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9a77fe700 1 -- 172.21.15.117:0/4069810408 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b401c470 con 0x7fb9a8005cb0 2023-01-25T06:49:16.337 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9c5668700 1 -- 172.21.15.117:0/4069810408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c006ead0 con 0x7fb9a8005cb0 2023-01-25T06:49:16.337 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9a77fe700 1 -- 172.21.15.117:0/4069810408 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9b401c750 con 0x7fb9a8005cb0 2023-01-25T06:49:16.337 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9c5668700 1 -- 172.21.15.117:0/4069810408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c006f700 con 0x7fb9a8005cb0 2023-01-25T06:49:16.338 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9a77fe700 1 -- 172.21.15.117:0/4069810408 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fb9b402d020 con 0x7fb9a8005cb0 2023-01-25T06:49:16.338 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.329+0000 7fb9a77fe700 1 -- 172.21.15.117:0/4069810408 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb9b401daa0 con 0x7fb9a8005cb0 2023-01-25T06:49:16.338 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.333+0000 7fb9c5668700 1 -- 172.21.15.117:0/4069810408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c0061570 con 0x7fb9a8005cb0 2023-01-25T06:49:16.340 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.337+0000 7fb9a77fe700 1 -- 172.21.15.117:0/4069810408 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7fb9b4017030 con 0x7fb9a8005cb0 2023-01-25T06:49:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.405+0000 7fb9c5668700 1 -- 172.21.15.117:0/4069810408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr_pool}] v 0) v1 -- 0x7fb9c00665d0 con 0x7fb9a8005cb0 2023-01-25T06:49:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.409+0000 7fb9a77fe700 1 -- 172.21.15.117:0/4069810408 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr_pool}]=0 v2)=0 v2) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb9b400f040 con 0x7fb9a8005cb0 2023-01-25T06:49:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.409+0000 7fb9c5668700 1 -- 172.21.15.117:0/4069810408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 msgr2=0x7fb9c01c5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:16.416 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.409+0000 7fb9c5668700 1 --2- 172.21.15.117:0/4069810408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9c01c5d00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4016040 tx=0x7fb9b40047e0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.416 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.413+0000 7fb9c5668700 1 -- 172.21.15.117:0/4069810408 shutdown_connections 2023-01-25T06:49:16.416 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.413+0000 7fb9c5668700 1 --2- 172.21.15.117:0/4069810408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a8005cb0 0x7fb9c01c5d00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.416 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.413+0000 7fb9c5668700 1 -- 172.21.15.117:0/4069810408 >> 172.21.15.117:0/4069810408 conn(0x7fb9c00a0db0 msgr2=0x7fb9c009f550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:16.416 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.413+0000 7fb9c5668700 1 -- 172.21.15.117:0/4069810408 shutdown_connections 2023-01-25T06:49:16.417 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.413+0000 7fb9c5668700 1 -- 172.21.15.117:0/4069810408 wait complete. 2023-01-25T06:49:16.427 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-25T06:49:16.427 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-25T06:49:16.428 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-25T06:49:16.430 INFO:tasks.ceph.mgr.x:Started 2023-01-25T06:49:16.431 INFO:tasks.ceph:Setting crush tunables to default 2023-01-25T06:49:16.431 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd crush tunables default 2023-01-25T06:49:16.542 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7feda5c50700 1 Processor -- start 2023-01-25T06:49:16.542 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7feda5c50700 1 -- start start 2023-01-25T06:49:16.544 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7feda5c50700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feda0097ef0 0x7feda0136cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:16.545 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7feda5c50700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda0097730 con 0x7feda009b0d0 2023-01-25T06:49:16.545 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7feda5c50700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda009be00 con 0x7feda0097ef0 2023-01-25T06:49:16.545 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7feda4c4e700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7feda009b0d0 0x7feda009b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:44370/0 (socket says 172.21.15.117:44370) 2023-01-25T06:49:16.545 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7feda4c4e700 1 -- 172.21.15.117:0/475098418 learned_addr learned my addr 172.21.15.117:0/475098418 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7fed9ffff700 1 --2- 172.21.15.117:0/475098418 >> v2:172.21.15.117:3300/0 conn(0x7feda0097ef0 0x7feda0136cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7fed9f7fe700 1 -- 172.21.15.117:0/475098418 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045335455 0 0) 0x7feda0097730 con 0x7feda009b0d0 2023-01-25T06:49:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.537+0000 7fed9f7fe700 1 -- 172.21.15.117:0/475098418 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed880035d0 con 0x7feda009b0d0 2023-01-25T06:49:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9ffff700 1 -- 172.21.15.117:0/475098418 >> v1:172.21.15.117:6789/0 conn(0x7feda009b0d0 legacy=0x7feda009b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:16.546 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9ffff700 1 -- 172.21.15.117:0/475098418 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda009bbb0 con 0x7feda0097ef0 2023-01-25T06:49:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9ffff700 1 --2- 172.21.15.117:0/475098418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 0x7feda0136cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed940097d0 tx=0x7fed94009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b1289baaffa734 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9f7fe700 1 -- 172.21.15.117:0/475098418 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed94004160 con 0x7feda0097ef0 2023-01-25T06:49:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9f7fe700 1 -- 172.21.15.117:0/475098418 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed940042c0 con 0x7feda0097ef0 2023-01-25T06:49:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9f7fe700 1 -- 172.21.15.117:0/475098418 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed94004580 con 0x7feda0097ef0 2023-01-25T06:49:16.547 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 -- 172.21.15.117:0/475098418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 msgr2=0x7feda0136cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 --2- 172.21.15.117:0/475098418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 0x7feda0136cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed940097d0 tx=0x7fed94009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 -- 172.21.15.117:0/475098418 shutdown_connections 2023-01-25T06:49:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 --2- 172.21.15.117:0/475098418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 0x7feda0136cb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 -- 172.21.15.117:0/475098418 >> 172.21.15.117:0/475098418 conn(0x7feda00a0120 msgr2=0x7feda00a04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:16.548 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 -- 172.21.15.117:0/475098418 shutdown_connections 2023-01-25T06:49:16.549 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 -- 172.21.15.117:0/475098418 wait complete. 2023-01-25T06:49:16.550 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 Processor -- start 2023-01-25T06:49:16.551 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 -- start start 2023-01-25T06:49:16.551 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 0x7feda01226b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:16.551 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda0132a80 con 0x7feda0097ef0 2023-01-25T06:49:16.551 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda4c4e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 0x7feda01226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:16.551 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda4c4e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 0x7feda01226b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33918/0 (socket says 172.21.15.117:33918) 2023-01-25T06:49:16.552 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda4c4e700 1 -- 172.21.15.117:0/825431266 learned_addr learned my addr 172.21.15.117:0/825431266 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:16.552 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda4c4e700 1 -- 172.21.15.117:0/825431266 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940094b0 con 0x7feda0097ef0 2023-01-25T06:49:16.552 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda4c4e700 1 --2- 172.21.15.117:0/825431266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 0x7feda01226b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed9800cf70 tx=0x7fed9800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:16.552 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9d7fa700 1 -- 172.21.15.117:0/825431266 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed98005de0 con 0x7feda0097ef0 2023-01-25T06:49:16.552 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9d7fa700 1 -- 172.21.15.117:0/825431266 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9800a8f0 con 0x7feda0097ef0 2023-01-25T06:49:16.553 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 -- 172.21.15.117:0/825431266 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda009aca0 con 0x7feda0097ef0 2023-01-25T06:49:16.553 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9d7fa700 1 -- 172.21.15.117:0/825431266 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed9800ab90 con 0x7feda0097ef0 2023-01-25T06:49:16.553 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7feda5c50700 1 -- 172.21.15.117:0/825431266 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda009c620 con 0x7feda0097ef0 2023-01-25T06:49:16.553 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9d7fa700 1 -- 172.21.15.117:0/825431266 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fed98019030 con 0x7feda0097ef0 2023-01-25T06:49:16.553 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.541+0000 7fed9d7fa700 1 -- 172.21.15.117:0/825431266 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fed9801c780 con 0x7feda0097ef0 2023-01-25T06:49:16.554 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.545+0000 7feda5c50700 1 -- 172.21.15.117:0/825431266 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda0097a60 con 0x7feda0097ef0 2023-01-25T06:49:16.554 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.545+0000 7fed9d7fa700 1 -- 172.21.15.117:0/825431266 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7fed98021060 con 0x7feda0097ef0 2023-01-25T06:49:16.619 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.613+0000 7feda5c50700 1 -- 172.21.15.117:0/825431266 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7feda0120b00 con 0x7feda0097ef0 2023-01-25T06:49:16.622 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.617+0000 7fed9d7fa700 1 -- 172.21.15.117:0/825431266 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fed98017070 con 0x7feda0097ef0 2023-01-25T06:49:16.623 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.621+0000 7feda5c50700 1 -- 172.21.15.117:0/825431266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 msgr2=0x7feda01226b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:16.624 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.621+0000 7feda5c50700 1 --2- 172.21.15.117:0/825431266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 0x7feda01226b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed9800cf70 tx=0x7fed9800a340 comp rx=0 tx=0).stop 2023-01-25T06:49:16.624 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.621+0000 7feda5c50700 1 -- 172.21.15.117:0/825431266 shutdown_connections 2023-01-25T06:49:16.624 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.621+0000 7feda5c50700 1 --2- 172.21.15.117:0/825431266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda0097ef0 0x7feda01226b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.624 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.621+0000 7feda5c50700 1 -- 172.21.15.117:0/825431266 >> 172.21.15.117:0/825431266 conn(0x7feda00a0120 msgr2=0x7feda0118780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:16.624 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.621+0000 7feda5c50700 1 -- 172.21.15.117:0/825431266 shutdown_connections 2023-01-25T06:49:16.625 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.621+0000 7feda5c50700 1 -- 172.21.15.117:0/825431266 wait complete. 2023-01-25T06:49:16.625 INFO:teuthology.orchestra.run.smithi117.stderr:adjusted tunables profile to default 2023-01-25T06:49:16.635 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-25T06:49:16.636 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:16.636 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-25T06:49:16.651 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:16.651 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-25T06:49:16.710 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-01-25T06:49:16.711 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-25T06:49:16.720 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 12f2c557-9775-4b67-a383-f8362a26e4fd 0 2023-01-25T06:49:16.741 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:16.737+0000 7f45b251d2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T06:49:16.824 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.817+0000 7f186bfff700 1 Processor -- start 2023-01-25T06:49:16.824 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.817+0000 7f186bfff700 1 -- start start 2023-01-25T06:49:16.825 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f186c134ac0 0x7f186c136ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:16.825 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186c09a110 con 0x7f186c12e7a0 2023-01-25T06:49:16.825 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c12e1a0 con 0x7f186c134ac0 2023-01-25T06:49:16.826 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186a7fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f186c134ac0 0x7f186c136ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:16.826 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186a7fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f186c134ac0 0x7f186c136ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33924/0 (socket says 172.21.15.117:33924) 2023-01-25T06:49:16.826 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186a7fc700 1 -- 172.21.15.117:0/3498023423 learned_addr learned my addr 172.21.15.117:0/3498023423 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:16.826 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186a7fc700 1 -- 172.21.15.117:0/3498023423 >> v1:172.21.15.117:6789/0 conn(0x7f186c12e7a0 legacy=0x7f186c12eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:16.827 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186a7fc700 1 -- 172.21.15.117:0/3498023423 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c12e390 con 0x7f186c134ac0 2023-01-25T06:49:16.827 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186a7fc700 1 --2- 172.21.15.117:0/3498023423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c134ac0 0x7f186c136ea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f185c009890 tx=0x7f185c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9434a5dbc819e929 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:16.827 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f1869ffb700 1 -- 172.21.15.117:0/3498023423 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f185c004160 con 0x7f186c134ac0 2023-01-25T06:49:16.827 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f1869ffb700 1 -- 172.21.15.117:0/3498023423 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c0042c0 con 0x7f186c134ac0 2023-01-25T06:49:16.827 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f1869ffb700 1 -- 172.21.15.117:0/3498023423 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f185c004580 con 0x7f186c134ac0 2023-01-25T06:49:16.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 -- 172.21.15.117:0/3498023423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c134ac0 msgr2=0x7f186c136ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:16.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 --2- 172.21.15.117:0/3498023423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c134ac0 0x7f186c136ea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f185c009890 tx=0x7f185c009ba0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 -- 172.21.15.117:0/3498023423 shutdown_connections 2023-01-25T06:49:16.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 --2- 172.21.15.117:0/3498023423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c134ac0 0x7f186c136ea0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 -- 172.21.15.117:0/3498023423 >> 172.21.15.117:0/3498023423 conn(0x7f186c0a4e30 msgr2=0x7f186c0698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 -- 172.21.15.117:0/3498023423 shutdown_connections 2023-01-25T06:49:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 -- 172.21.15.117:0/3498023423 wait complete. 2023-01-25T06:49:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 Processor -- start 2023-01-25T06:49:16.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 -- start start 2023-01-25T06:49:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c12e7a0 0x7f186c0a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.821+0000 7f186bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c09f950 con 0x7f186c12e7a0 2023-01-25T06:49:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f186affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c12e7a0 0x7f186c0a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f186affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c12e7a0 0x7f186c0a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33926/0 (socket says 172.21.15.117:33926) 2023-01-25T06:49:16.832 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f186affd700 1 -- 172.21.15.117:0/1152866778 learned_addr learned my addr 172.21.15.117:0/1152866778 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:16.833 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f186affd700 1 -- 172.21.15.117:0/1152866778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009570 con 0x7f186c12e7a0 2023-01-25T06:49:16.833 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f186affd700 1 --2- 172.21.15.117:0/1152866778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c12e7a0 0x7f186c0a12f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f186000c480 tx=0x7f186000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:16.833 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f1853fff700 1 -- 172.21.15.117:0/1152866778 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1860005150 con 0x7f186c12e7a0 2023-01-25T06:49:16.833 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f1853fff700 1 -- 172.21.15.117:0/1152866778 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18600052b0 con 0x7f186c12e7a0 2023-01-25T06:49:16.833 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f186bfff700 1 -- 172.21.15.117:0/1152866778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c09acb0 con 0x7f186c12e7a0 2023-01-25T06:49:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f1853fff700 1 -- 172.21.15.117:0/1152866778 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1860005590 con 0x7f186c12e7a0 2023-01-25T06:49:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f186bfff700 1 -- 172.21.15.117:0/1152866778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c09a790 con 0x7f186c12e7a0 2023-01-25T06:49:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f1853fff700 1 -- 172.21.15.117:0/1152866778 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f1860010050 con 0x7f186c12e7a0 2023-01-25T06:49:16.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.825+0000 7f1853fff700 1 -- 172.21.15.117:0/1152866778 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1860007570 con 0x7f186c12e7a0 2023-01-25T06:49:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.829+0000 7f186bfff700 1 -- 172.21.15.117:0/1152866778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c09a4f0 con 0x7f186c12e7a0 2023-01-25T06:49:16.835 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.829+0000 7f1853fff700 1 -- 172.21.15.117:0/1152866778 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f1860007030 con 0x7f186c12e7a0 2023-01-25T06:49:16.909 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.905+0000 7f186bfff700 1 -- 172.21.15.117:0/1152866778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "12f2c557-9775-4b67-a383-f8362a26e4fd", "id": 0} v 0) v1 -- 0x7f186c04ebb0 con 0x7f186c12e7a0 2023-01-25T06:49:16.913 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.909+0000 7f1853fff700 1 -- 172.21.15.117:0/1152866778 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "12f2c557-9775-4b67-a383-f8362a26e4fd", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f18600069c0 con 0x7f186c12e7a0 2023-01-25T06:49:16.914 INFO:teuthology.orchestra.run.smithi117.stdout:0 2023-01-25T06:49:16.915 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.909+0000 7f186bfff700 1 -- 172.21.15.117:0/1152866778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c12e7a0 msgr2=0x7f186c0a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:16.915 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.909+0000 7f186bfff700 1 --2- 172.21.15.117:0/1152866778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c12e7a0 0x7f186c0a12f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f186000c480 tx=0x7f186000c840 comp rx=0 tx=0).stop 2023-01-25T06:49:16.916 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.909+0000 7f186bfff700 1 -- 172.21.15.117:0/1152866778 shutdown_connections 2023-01-25T06:49:16.916 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.909+0000 7f186bfff700 1 --2- 172.21.15.117:0/1152866778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c12e7a0 0x7f186c0a12f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:16.916 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.909+0000 7f186bfff700 1 -- 172.21.15.117:0/1152866778 >> 172.21.15.117:0/1152866778 conn(0x7f186c0a4e30 msgr2=0x7f186c12cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:16.916 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.909+0000 7f186bfff700 1 -- 172.21.15.117:0/1152866778 shutdown_connections 2023-01-25T06:49:16.917 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:16.913+0000 7f186bfff700 1 -- 172.21.15.117:0/1152866778 wait complete. 2023-01-25T06:49:16.930 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new dd866452-4e42-4fdd-8635-12cead61e275 1 2023-01-25T06:49:17.083 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.077+0000 7f321dca4700 1 Processor -- start 2023-01-25T06:49:17.084 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.077+0000 7f321dca4700 1 -- start start 2023-01-25T06:49:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3218129580 0x7f3218129950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321809cad0 con 0x7f321812f880 2023-01-25T06:49:17.085 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321806fa30 con 0x7f3218129580 2023-01-25T06:49:17.086 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321cca2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3218129580 0x7f3218129950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:17.086 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f3217fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f321812f880 0x7f321812fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:44400/0 (socket says 172.21.15.117:44400) 2023-01-25T06:49:17.087 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f3217fff700 1 -- 172.21.15.117:0/1824245467 learned_addr learned my addr 172.21.15.117:0/1824245467 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:17.087 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321cca2700 1 -- 172.21.15.117:0/1824245467 >> v1:172.21.15.117:6789/0 conn(0x7f321812f880 legacy=0x7f321812fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:17.087 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321cca2700 1 -- 172.21.15.117:0/1824245467 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321809af30 con 0x7f3218129580 2023-01-25T06:49:17.087 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f32177fe700 1 -- 172.21.15.117:0/1824245467 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282844164 0 0) 0x7f321809cad0 con 0x7f321812f880 2023-01-25T06:49:17.088 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321cca2700 1 --2- 172.21.15.117:0/1824245467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 0x7f3218129950 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3210009d90 tx=0x7f321000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1300ae5cd799b6d server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:17.088 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f32177fe700 1 -- 172.21.15.117:0/1824245467 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f321000e070 con 0x7f3218129580 2023-01-25T06:49:17.088 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f32177fe700 1 -- 172.21.15.117:0/1824245467 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321000bc80 con 0x7f3218129580 2023-01-25T06:49:17.088 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f32177fe700 1 -- 172.21.15.117:0/1824245467 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3210003a40 con 0x7f3218129580 2023-01-25T06:49:17.091 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 -- 172.21.15.117:0/1824245467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 msgr2=0x7f3218129950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:17.091 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 --2- 172.21.15.117:0/1824245467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 0x7f3218129950 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3210009d90 tx=0x7f321000b330 comp rx=0 tx=0).stop 2023-01-25T06:49:17.091 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 -- 172.21.15.117:0/1824245467 shutdown_connections 2023-01-25T06:49:17.091 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 --2- 172.21.15.117:0/1824245467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 0x7f3218129950 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:17.091 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 -- 172.21.15.117:0/1824245467 >> 172.21.15.117:0/1824245467 conn(0x7f321809d970 msgr2=0x7f321809dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:17.092 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 -- 172.21.15.117:0/1824245467 shutdown_connections 2023-01-25T06:49:17.092 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.081+0000 7f321dca4700 1 -- 172.21.15.117:0/1824245467 wait complete. 2023-01-25T06:49:17.092 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321dca4700 1 Processor -- start 2023-01-25T06:49:17.092 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321dca4700 1 -- start start 2023-01-25T06:49:17.093 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321dca4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 0x7f3218127b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:17.093 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321dca4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3218098e20 con 0x7f3218129580 2023-01-25T06:49:17.093 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321cca2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 0x7f3218127b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:17.093 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321cca2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 0x7f3218127b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33942/0 (socket says 172.21.15.117:33942) 2023-01-25T06:49:17.093 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321cca2700 1 -- 172.21.15.117:0/2790248342 learned_addr learned my addr 172.21.15.117:0/2790248342 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321cca2700 1 -- 172.21.15.117:0/2790248342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3210009a70 con 0x7f3218129580 2023-01-25T06:49:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321cca2700 1 --2- 172.21.15.117:0/2790248342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 0x7f3218127b60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3210009470 tx=0x7f3210004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f32157fa700 1 -- 172.21.15.117:0/2790248342 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3210004750 con 0x7f3218129580 2023-01-25T06:49:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f32157fa700 1 -- 172.21.15.117:0/2790248342 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32100048d0 con 0x7f3218129580 2023-01-25T06:49:17.094 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321dca4700 1 -- 172.21.15.117:0/2790248342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3218070460 con 0x7f3218129580 2023-01-25T06:49:17.095 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f32157fa700 1 -- 172.21.15.117:0/2790248342 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32100125a0 con 0x7f3218129580 2023-01-25T06:49:17.095 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f321dca4700 1 -- 172.21.15.117:0/2790248342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321806fcf0 con 0x7f3218129580 2023-01-25T06:49:17.095 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f32157fa700 1 -- 172.21.15.117:0/2790248342 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3210012700 con 0x7f3218129580 2023-01-25T06:49:17.095 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.085+0000 7f32157fa700 1 -- 172.21.15.117:0/2790248342 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f321001dd70 con 0x7f3218129580 2023-01-25T06:49:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.089+0000 7f321dca4700 1 -- 172.21.15.117:0/2790248342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321804ebb0 con 0x7f3218129580 2023-01-25T06:49:17.096 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.089+0000 7f32157fa700 1 -- 172.21.15.117:0/2790248342 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f321001d4d0 con 0x7f3218129580 2023-01-25T06:49:17.171 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.165+0000 7f321dca4700 1 -- 172.21.15.117:0/2790248342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dd866452-4e42-4fdd-8635-12cead61e275", "id": 1} v 0) v1 -- 0x7f3218098fa0 con 0x7f3218129580 2023-01-25T06:49:17.174 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.169+0000 7f32157fa700 1 -- 172.21.15.117:0/2790248342 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd866452-4e42-4fdd-8635-12cead61e275", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f321002e3f0 con 0x7f3218129580 2023-01-25T06:49:17.174 INFO:teuthology.orchestra.run.smithi117.stdout:1 2023-01-25T06:49:17.175 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.169+0000 7f321dca4700 1 -- 172.21.15.117:0/2790248342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 msgr2=0x7f3218127b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:17.175 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.169+0000 7f321dca4700 1 --2- 172.21.15.117:0/2790248342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 0x7f3218127b60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3210009470 tx=0x7f3210004af0 comp rx=0 tx=0).stop 2023-01-25T06:49:17.175 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.169+0000 7f321dca4700 1 -- 172.21.15.117:0/2790248342 shutdown_connections 2023-01-25T06:49:17.176 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.169+0000 7f321dca4700 1 --2- 172.21.15.117:0/2790248342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3218129580 0x7f3218127b60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:17.176 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.169+0000 7f321dca4700 1 -- 172.21.15.117:0/2790248342 >> 172.21.15.117:0/2790248342 conn(0x7f321809d970 msgr2=0x7f321809f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:17.176 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.169+0000 7f321dca4700 1 -- 172.21.15.117:0/2790248342 shutdown_connections 2023-01-25T06:49:17.176 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.173+0000 7f321dca4700 1 -- 172.21.15.117:0/2790248342 wait complete. 2023-01-25T06:49:17.187 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new ede1a6e7-9acf-4606-835e-e3518ce94df6 2 2023-01-25T06:49:17.297 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4ca5396700 1 Processor -- start 2023-01-25T06:49:17.297 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4ca5396700 1 -- start start 2023-01-25T06:49:17.300 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4ca5396700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4ca009c7c0 0x7f4ca0130230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:17.300 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4ca5396700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca00718f0 con 0x7f4ca012fe60 2023-01-25T06:49:17.300 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4ca5396700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca012fa10 con 0x7f4ca009c7c0 2023-01-25T06:49:17.301 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9ffff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4ca012fe60 0x7f4ca00992c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:44414/0 (socket says 172.21.15.117:44414) 2023-01-25T06:49:17.301 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9ffff700 1 -- 172.21.15.117:0/3892327518 learned_addr learned my addr 172.21.15.117:0/3892327518 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:17.301 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9f7fe700 1 --2- 172.21.15.117:0/3892327518 >> v2:172.21.15.117:3300/0 conn(0x7f4ca009c7c0 0x7f4ca0130230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:17.301 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9effd700 1 -- 172.21.15.117:0/3892327518 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637606003 0 0) 0x7f4ca00718f0 con 0x7f4ca012fe60 2023-01-25T06:49:17.302 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9effd700 1 -- 172.21.15.117:0/3892327518 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c880035d0 con 0x7f4ca012fe60 2023-01-25T06:49:17.302 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9f7fe700 1 -- 172.21.15.117:0/3892327518 >> v1:172.21.15.117:6789/0 conn(0x7f4ca012fe60 legacy=0x7f4ca00992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:17.302 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9f7fe700 1 -- 172.21.15.117:0/3892327518 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca012f700 con 0x7f4ca009c7c0 2023-01-25T06:49:17.302 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9f7fe700 1 --2- 172.21.15.117:0/3892327518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 0x7f4ca0130230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4c940097c0 tx=0x7f4c94009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7151dd41a5e435ea server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:17.302 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9effd700 1 -- 172.21.15.117:0/3892327518 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c94005d80 con 0x7f4ca009c7c0 2023-01-25T06:49:17.303 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9effd700 1 -- 172.21.15.117:0/3892327518 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c94004030 con 0x7f4ca009c7c0 2023-01-25T06:49:17.303 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.293+0000 7f4c9effd700 1 -- 172.21.15.117:0/3892327518 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c940042d0 con 0x7f4ca009c7c0 2023-01-25T06:49:17.303 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 -- 172.21.15.117:0/3892327518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 msgr2=0x7f4ca0130230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:17.303 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 --2- 172.21.15.117:0/3892327518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 0x7f4ca0130230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4c940097c0 tx=0x7f4c94009ad0 comp rx=0 tx=0).stop 2023-01-25T06:49:17.303 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 -- 172.21.15.117:0/3892327518 shutdown_connections 2023-01-25T06:49:17.304 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 --2- 172.21.15.117:0/3892327518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 0x7f4ca0130230 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:17.304 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 -- 172.21.15.117:0/3892327518 >> 172.21.15.117:0/3892327518 conn(0x7f4ca006a750 msgr2=0x7f4ca0070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:17.304 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 -- 172.21.15.117:0/3892327518 shutdown_connections 2023-01-25T06:49:17.304 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 -- 172.21.15.117:0/3892327518 wait complete. 2023-01-25T06:49:17.306 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 Processor -- start 2023-01-25T06:49:17.306 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 -- start start 2023-01-25T06:49:17.306 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 0x7f4ca01c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:17.307 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca009e570 con 0x7f4ca009c7c0 2023-01-25T06:49:17.307 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 0x7f4ca01c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:17.307 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 0x7f4ca01c3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33962/0 (socket says 172.21.15.117:33962) 2023-01-25T06:49:17.307 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9ffff700 1 -- 172.21.15.117:0/2276693507 learned_addr learned my addr 172.21.15.117:0/2276693507 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:17.308 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9ffff700 1 -- 172.21.15.117:0/2276693507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94009450 con 0x7f4ca009c7c0 2023-01-25T06:49:17.308 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9ffff700 1 --2- 172.21.15.117:0/2276693507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 0x7f4ca01c3ce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c470 tx=0x7f4c9000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:17.308 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9cff9700 1 -- 172.21.15.117:0/2276693507 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c90005cc0 con 0x7f4ca009c7c0 2023-01-25T06:49:17.308 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9cff9700 1 -- 172.21.15.117:0/2276693507 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90005e20 con 0x7f4ca009c7c0 2023-01-25T06:49:17.308 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 -- 172.21.15.117:0/2276693507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca009ba40 con 0x7f4ca009c7c0 2023-01-25T06:49:17.309 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9cff9700 1 -- 172.21.15.117:0/2276693507 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c90007220 con 0x7f4ca009c7c0 2023-01-25T06:49:17.309 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4ca5396700 1 -- 172.21.15.117:0/2276693507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca009cdf0 con 0x7f4ca009c7c0 2023-01-25T06:49:17.309 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9cff9700 1 -- 172.21.15.117:0/2276693507 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4c90019030 con 0x7f4ca009c7c0 2023-01-25T06:49:17.309 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.297+0000 7f4c9cff9700 1 -- 172.21.15.117:0/2276693507 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f4c9001b7d0 con 0x7f4ca009c7c0 2023-01-25T06:49:17.309 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.301+0000 7f4ca5396700 1 -- 172.21.15.117:0/2276693507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c80005100 con 0x7f4ca009c7c0 2023-01-25T06:49:17.310 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.305+0000 7f4c9cff9700 1 -- 172.21.15.117:0/2276693507 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f4c90017070 con 0x7f4ca009c7c0 2023-01-25T06:49:17.385 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.381+0000 7f4ca5396700 1 -- 172.21.15.117:0/2276693507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ede1a6e7-9acf-4606-835e-e3518ce94df6", "id": 2} v 0) v1 -- 0x7f4c800056b0 con 0x7f4ca009c7c0 2023-01-25T06:49:17.388 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.385+0000 7f4c9cff9700 1 -- 172.21.15.117:0/2276693507 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ede1a6e7-9acf-4606-835e-e3518ce94df6", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4c9001c510 con 0x7f4ca009c7c0 2023-01-25T06:49:17.389 INFO:teuthology.orchestra.run.smithi117.stdout:2 2023-01-25T06:49:17.390 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.385+0000 7f4ca5396700 1 -- 172.21.15.117:0/2276693507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 msgr2=0x7f4ca01c3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:17.390 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.385+0000 7f4ca5396700 1 --2- 172.21.15.117:0/2276693507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 0x7f4ca01c3ce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c470 tx=0x7f4c9000c830 comp rx=0 tx=0).stop 2023-01-25T06:49:17.391 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.385+0000 7f4ca5396700 1 -- 172.21.15.117:0/2276693507 shutdown_connections 2023-01-25T06:49:17.391 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.385+0000 7f4ca5396700 1 --2- 172.21.15.117:0/2276693507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ca009c7c0 0x7f4ca01c3ce0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:17.391 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.385+0000 7f4ca5396700 1 -- 172.21.15.117:0/2276693507 >> 172.21.15.117:0/2276693507 conn(0x7f4ca006a750 msgr2=0x7f4ca01b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:17.391 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.385+0000 7f4ca5396700 1 -- 172.21.15.117:0/2276693507 shutdown_connections 2023-01-25T06:49:17.392 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.385+0000 7f4ca5396700 1 -- 172.21.15.117:0/2276693507 wait complete. 2023-01-25T06:49:17.404 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-25T06:49:17.405 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-25T06:49:17.407 INFO:tasks.ceph.osd.0:Started 2023-01-25T06:49:17.408 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-25T06:49:17.408 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-25T06:49:17.410 INFO:tasks.ceph.osd.1:Started 2023-01-25T06:49:17.411 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-25T06:49:17.411 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-25T06:49:17.413 INFO:tasks.ceph.osd.2:Started 2023-01-25T06:49:17.414 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T06:49:17.422 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:17.417+0000 7f45b251d2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T06:49:17.491 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:17.485+0000 7f45b251d2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T06:49:17.530 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.525+0000 7fed108fe700 1 Processor -- start 2023-01-25T06:49:17.530 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.525+0000 7fed108fe700 1 -- start start 2023-01-25T06:49:17.531 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.525+0000 7fed108fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fed0c065330 0x7fed0c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:17.532 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.525+0000 7fed108fe700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed0c12e590 con 0x7fed0c12eb00 2023-01-25T06:49:17.532 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.525+0000 7fed108fe700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0c065700 con 0x7fed0c065330 2023-01-25T06:49:17.532 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.525+0000 7fed0b7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fed0c065330 0x7fed0c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:17.532 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.525+0000 7fed02ffd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fed0c12eb00 0x7fed0c12ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:44456/0 (socket says 172.21.15.117:44456) 2023-01-25T06:49:17.533 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.525+0000 7fed0b7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fed0c065330 0x7fed0c06efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33990/0 (socket says 172.21.15.117:33990) 2023-01-25T06:49:17.533 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.525+0000 7fed02ffd700 1 -- 172.21.15.117:0/2927616923 learned_addr learned my addr 172.21.15.117:0/2927616923 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:17.533 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed0b7fe700 1 -- 172.21.15.117:0/2927616923 >> v1:172.21.15.117:6789/0 conn(0x7fed0c12eb00 legacy=0x7fed0c12ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:17.533 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed0b7fe700 1 -- 172.21.15.117:0/2927616923 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c12e7f0 con 0x7fed0c065330 2023-01-25T06:49:17.533 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed0b7fe700 1 --2- 172.21.15.117:0/2927616923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 0x7fed0c06efb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0097d0 tx=0x7fecfc009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17457f1f4a24373 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:17.535 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed0affd700 1 -- 172.21.15.117:0/2927616923 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecfc0041c0 con 0x7fed0c065330 2023-01-25T06:49:17.535 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed0affd700 1 -- 172.21.15.117:0/2927616923 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc004320 con 0x7fed0c065330 2023-01-25T06:49:17.535 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed0affd700 1 -- 172.21.15.117:0/2927616923 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecfc0036c0 con 0x7fed0c065330 2023-01-25T06:49:17.535 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed108fe700 1 -- 172.21.15.117:0/2927616923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 msgr2=0x7fed0c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed108fe700 1 --2- 172.21.15.117:0/2927616923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 0x7fed0c06efb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0097d0 tx=0x7fecfc009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed108fe700 1 -- 172.21.15.117:0/2927616923 shutdown_connections 2023-01-25T06:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed108fe700 1 --2- 172.21.15.117:0/2927616923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 0x7fed0c06efb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed108fe700 1 -- 172.21.15.117:0/2927616923 >> 172.21.15.117:0/2927616923 conn(0x7fed0c09d470 msgr2=0x7fed0c09d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:17.536 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed108fe700 1 -- 172.21.15.117:0/2927616923 shutdown_connections 2023-01-25T06:49:17.537 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.529+0000 7fed108fe700 1 -- 172.21.15.117:0/2927616923 wait complete. 2023-01-25T06:49:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed108fe700 1 Processor -- start 2023-01-25T06:49:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed108fe700 1 -- start start 2023-01-25T06:49:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed108fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 0x7fed0c099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:17.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed108fe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c098a80 con 0x7fed0c065330 2023-01-25T06:49:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed0b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 0x7fed0c099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed0b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 0x7fed0c099d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34000/0 (socket says 172.21.15.117:34000) 2023-01-25T06:49:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed0b7fe700 1 -- 172.21.15.117:0/1693446064 learned_addr learned my addr 172.21.15.117:0/1693446064 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed0b7fe700 1 -- 172.21.15.117:0/1693446064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc0094b0 con 0x7fed0c065330 2023-01-25T06:49:17.540 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed0b7fe700 1 --2- 172.21.15.117:0/1693446064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 0x7fed0c099d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009b20 tx=0x7fecfc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:17.541 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed08ff9700 1 -- 172.21.15.117:0/1693446064 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecfc006ea0 con 0x7fed0c065330 2023-01-25T06:49:17.541 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed08ff9700 1 -- 172.21.15.117:0/1693446064 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc011630 con 0x7fed0c065330 2023-01-25T06:49:17.541 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed108fe700 1 -- 172.21.15.117:0/1693446064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0c09e870 con 0x7fed0c065330 2023-01-25T06:49:17.541 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed08ff9700 1 -- 172.21.15.117:0/1693446064 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecfc0118d0 con 0x7fed0c065330 2023-01-25T06:49:17.541 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed108fe700 1 -- 172.21.15.117:0/1693446064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c06e610 con 0x7fed0c065330 2023-01-25T06:49:17.542 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed08ff9700 1 -- 172.21.15.117:0/1693446064 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fecfc00f050 con 0x7fed0c065330 2023-01-25T06:49:17.542 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed08ff9700 1 -- 172.21.15.117:0/1693446064 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fecfc01de30 con 0x7fed0c065330 2023-01-25T06:49:17.542 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.533+0000 7fed108fe700 1 -- 172.21.15.117:0/1693446064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0c09ea00 con 0x7fed0c065330 2023-01-25T06:49:17.543 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.537+0000 7fed08ff9700 1 -- 172.21.15.117:0/1693446064 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7fecfc021020 con 0x7fed0c065330 2023-01-25T06:49:17.603 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.601+0000 7fed108fe700 1 -- 172.21.15.117:0/1693446064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed0c09a660 con 0x7fed0c065330 2023-01-25T06:49:17.604 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.601+0000 7fed08ff9700 1 -- 172.21.15.117:0/1693446064 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v5) v1 ==== 74+0+3404 (secure 0 0 0) 0x7fecfc016070 con 0x7fed0c065330 2023-01-25T06:49:17.605 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:17.605 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"fsid":"52037823-4da1-437a-aebe-937b909d2d65","created":"2023-01-25T06:49:16.306989+0000","modified":"2023-01-25T06:49:17.386973+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T06:49:17.386973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"12f2c557-9775-4b67-a383-f8362a26e4fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd866452-4e42-4fdd-8635-12cead61e275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ede1a6e7-9acf-4606-835e-e3518ce94df6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:49:17.606 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.601+0000 7fed108fe700 1 -- 172.21.15.117:0/1693446064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 msgr2=0x7fed0c099d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:17.606 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.601+0000 7fed108fe700 1 --2- 172.21.15.117:0/1693446064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 0x7fed0c099d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009b20 tx=0x7fecfc004770 comp rx=0 tx=0).stop 2023-01-25T06:49:17.606 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.601+0000 7fed108fe700 1 -- 172.21.15.117:0/1693446064 shutdown_connections 2023-01-25T06:49:17.606 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.601+0000 7fed108fe700 1 --2- 172.21.15.117:0/1693446064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed0c065330 0x7fed0c099d50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:17.607 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.601+0000 7fed108fe700 1 -- 172.21.15.117:0/1693446064 >> 172.21.15.117:0/1693446064 conn(0x7fed0c09d470 msgr2=0x7fed0c132dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:17.607 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.601+0000 7fed108fe700 1 -- 172.21.15.117:0/1693446064 shutdown_connections 2023-01-25T06:49:17.607 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:17.601+0000 7fed108fe700 1 -- 172.21.15.117:0/1693446064 wait complete. 2023-01-25T06:49:17.617 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-25T06:49:17.618 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-25T06:49:17.621 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:17.617+0000 7f45b251d2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T06:49:17.777 INFO:tasks.ceph.osd.0.smithi117.stderr:2023-01-25T06:49:17.773+0000 7fe4b4235680 -1 Falling back to public interface 2023-01-25T06:49:17.784 INFO:tasks.ceph.osd.1.smithi117.stderr:2023-01-25T06:49:17.781+0000 7f38882d7680 -1 Falling back to public interface 2023-01-25T06:49:17.813 INFO:tasks.ceph.osd.2.smithi117.stderr:2023-01-25T06:49:17.809+0000 7f47fc83c680 -1 Falling back to public interface 2023-01-25T06:49:17.861 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:17.857+0000 7f45b251d2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T06:49:17.913 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:17.909+0000 7f45b251d2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T06:49:17.920 DEBUG:teuthology.orchestra.run.smithi117:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T06:49:18.020 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05f5abe700 1 Processor -- start 2023-01-25T06:49:18.020 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05f5abe700 1 -- start start 2023-01-25T06:49:18.021 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05f5abe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05f00972a0 0x7f05f0097670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:18.021 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05f5abe700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f0098c20 con 0x7f05f0097bb0 2023-01-25T06:49:18.021 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05f5abe700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f0071910 con 0x7f05f00972a0 2023-01-25T06:49:18.021 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05f4abc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05f00972a0 0x7f05f0097670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:18.021 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05effff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f05f0097bb0 0x7f05f012e890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:44458/0 (socket says 172.21.15.117:44458) 2023-01-25T06:49:18.022 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05effff700 1 -- 172.21.15.117:0/929390655 learned_addr learned my addr 172.21.15.117:0/929390655 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:18.022 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05f4abc700 1 -- 172.21.15.117:0/929390655 >> v1:172.21.15.117:6789/0 conn(0x7f05f0097bb0 legacy=0x7f05f012e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:18.022 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05f4abc700 1 -- 172.21.15.117:0/929390655 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f0098e50 con 0x7f05f00972a0 2023-01-25T06:49:18.023 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05f4abc700 1 --2- 172.21.15.117:0/929390655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 0x7f05f0097670 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f05e00097d0 tx=0x7f05e0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7227d1255fc0c03 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:18.023 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.013+0000 7f05ef7fe700 1 -- 172.21.15.117:0/929390655 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05e00041c0 con 0x7f05f00972a0 2023-01-25T06:49:18.023 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05ef7fe700 1 -- 172.21.15.117:0/929390655 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e0004320 con 0x7f05f00972a0 2023-01-25T06:49:18.024 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05ef7fe700 1 -- 172.21.15.117:0/929390655 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05e00036c0 con 0x7f05f00972a0 2023-01-25T06:49:18.024 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 -- 172.21.15.117:0/929390655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 msgr2=0x7f05f0097670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:18.025 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 --2- 172.21.15.117:0/929390655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 0x7f05f0097670 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f05e00097d0 tx=0x7f05e0009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:18.025 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 -- 172.21.15.117:0/929390655 shutdown_connections 2023-01-25T06:49:18.025 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 --2- 172.21.15.117:0/929390655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 0x7f05f0097670 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:18.026 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 -- 172.21.15.117:0/929390655 >> 172.21.15.117:0/929390655 conn(0x7f05f009cb20 msgr2=0x7f05f009cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:18.026 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 -- 172.21.15.117:0/929390655 shutdown_connections 2023-01-25T06:49:18.027 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 -- 172.21.15.117:0/929390655 wait complete. 2023-01-25T06:49:18.029 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 Processor -- start 2023-01-25T06:49:18.030 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 -- start start 2023-01-25T06:49:18.030 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 0x7f05f012eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:18.030 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f5abe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f011d7f0 con 0x7f05f00972a0 2023-01-25T06:49:18.030 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f4abc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 0x7f05f012eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:18.031 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f4abc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 0x7f05f012eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34026/0 (socket says 172.21.15.117:34026) 2023-01-25T06:49:18.031 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.017+0000 7f05f4abc700 1 -- 172.21.15.117:0/3780161240 learned_addr learned my addr 172.21.15.117:0/3780161240 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:18.031 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.021+0000 7f05f4abc700 1 -- 172.21.15.117:0/3780161240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00094b0 con 0x7f05f00972a0 2023-01-25T06:49:18.031 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.021+0000 7f05f4abc700 1 --2- 172.21.15.117:0/3780161240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 0x7f05f012eb00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009b20 tx=0x7f05e0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:18.032 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.021+0000 7f05ed7fa700 1 -- 172.21.15.117:0/3780161240 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05e0006ea0 con 0x7f05f00972a0 2023-01-25T06:49:18.032 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.021+0000 7f05ed7fa700 1 -- 172.21.15.117:0/3780161240 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e0011630 con 0x7f05f00972a0 2023-01-25T06:49:18.032 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.021+0000 7f05f5abe700 1 -- 172.21.15.117:0/3780161240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f00983d0 con 0x7f05f00972a0 2023-01-25T06:49:18.032 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.021+0000 7f05ed7fa700 1 -- 172.21.15.117:0/3780161240 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05e00118d0 con 0x7f05f00972a0 2023-01-25T06:49:18.033 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.021+0000 7f05f5abe700 1 -- 172.21.15.117:0/3780161240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f00992c0 con 0x7f05f00972a0 2023-01-25T06:49:18.033 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.021+0000 7f05ed7fa700 1 -- 172.21.15.117:0/3780161240 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f05e000f050 con 0x7f05f00972a0 2023-01-25T06:49:18.033 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.021+0000 7f05ed7fa700 1 -- 172.21.15.117:0/3780161240 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f05e00063d0 con 0x7f05f00972a0 2023-01-25T06:49:18.034 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.025+0000 7f05f5abe700 1 -- 172.21.15.117:0/3780161240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f009a420 con 0x7f05f00972a0 2023-01-25T06:49:18.034 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.029+0000 7f05ed7fa700 1 -- 172.21.15.117:0/3780161240 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f05e0021020 con 0x7f05f00972a0 2023-01-25T06:49:18.098 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.093+0000 7f05f5abe700 1 -- 172.21.15.117:0/3780161240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05f012f7d0 con 0x7f05f00972a0 2023-01-25T06:49:18.099 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.093+0000 7f05ed7fa700 1 -- 172.21.15.117:0/3780161240 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v5) v1 ==== 74+0+3404 (secure 0 0 0) 0x7f05e0016070 con 0x7f05f00972a0 2023-01-25T06:49:18.099 INFO:teuthology.misc.health.smithi117.stdout: 2023-01-25T06:49:18.099 INFO:teuthology.misc.health.smithi117.stdout:{"epoch":5,"fsid":"52037823-4da1-437a-aebe-937b909d2d65","created":"2023-01-25T06:49:16.306989+0000","modified":"2023-01-25T06:49:17.386973+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T06:49:17.386973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"12f2c557-9775-4b67-a383-f8362a26e4fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd866452-4e42-4fdd-8635-12cead61e275","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ede1a6e7-9acf-4606-835e-e3518ce94df6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:49:18.100 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.097+0000 7f05f5abe700 1 -- 172.21.15.117:0/3780161240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 msgr2=0x7f05f012eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:18.101 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.097+0000 7f05f5abe700 1 --2- 172.21.15.117:0/3780161240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 0x7f05f012eb00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009b20 tx=0x7f05e0004770 comp rx=0 tx=0).stop 2023-01-25T06:49:18.101 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.097+0000 7f05f5abe700 1 -- 172.21.15.117:0/3780161240 shutdown_connections 2023-01-25T06:49:18.101 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.097+0000 7f05f5abe700 1 --2- 172.21.15.117:0/3780161240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05f00972a0 0x7f05f012eb00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:18.101 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.097+0000 7f05f5abe700 1 -- 172.21.15.117:0/3780161240 >> 172.21.15.117:0/3780161240 conn(0x7f05f009cb20 msgr2=0x7f05f01b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:18.102 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.097+0000 7f05f5abe700 1 -- 172.21.15.117:0/3780161240 shutdown_connections 2023-01-25T06:49:18.102 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:18.097+0000 7f05f5abe700 1 -- 172.21.15.117:0/3780161240 wait complete. 2023-01-25T06:49:18.106 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:18.101+0000 7f45b251d2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T06:49:18.110 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-01-25T06:49:18.174 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:18.169+0000 7f45b251d2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T06:49:18.240 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:18.237+0000 7f45b251d2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T06:49:18.669 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:18.665+0000 7f45b251d2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T06:49:18.792 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:18.789+0000 7f45b251d2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T06:49:19.110 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:19.105+0000 7f45b251d2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T06:49:19.276 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:19.273+0000 7f45b251d2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T06:49:19.360 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:19.357+0000 7f45b251d2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T06:49:19.415 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:19.413+0000 7f45b251d2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T06:49:20.254 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:20.249+0000 7f45b251d2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T06:49:20.319 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:20.317+0000 7f45b251d2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T06:49:20.497 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:20.493+0000 7f45b251d2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T06:49:20.559 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:20.553+0000 7f45b251d2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T06:49:20.597 INFO:tasks.ceph.osd.1.smithi117.stderr:2023-01-25T06:49:20.593+0000 7f38882d7680 -1 osd.1 0 log_to_monitors true 2023-01-25T06:49:20.631 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:20.625+0000 7f45b251d2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T06:49:20.808 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:20.805+0000 7f45b251d2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T06:49:20.828 INFO:tasks.ceph.osd.0.smithi117.stderr:2023-01-25T06:49:20.825+0000 7fe4b4235680 -1 osd.0 0 log_to_monitors true 2023-01-25T06:49:20.877 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:20.873+0000 7f45b251d2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T06:49:20.947 INFO:tasks.ceph.osd.2.smithi117.stderr:2023-01-25T06:49:20.941+0000 7f47fc83c680 -1 osd.2 0 log_to_monitors true 2023-01-25T06:49:21.658 INFO:tasks.ceph.osd.1.smithi117.stderr:2023-01-25T06:49:21.653+0000 7f3884114700 -1 osd.1 0 waiting for initial osdmap 2023-01-25T06:49:21.665 INFO:tasks.ceph.osd.1.smithi117.stderr:2023-01-25T06:49:21.661+0000 7f387fbcb700 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:49:21.680 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:21.677+0000 7f45b251d2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T06:49:22.669 INFO:tasks.ceph.osd.0.smithi117.stderr:2023-01-25T06:49:22.665+0000 7fe4b0072700 -1 osd.0 0 waiting for initial osdmap 2023-01-25T06:49:22.670 INFO:tasks.ceph.osd.2.smithi117.stderr:2023-01-25T06:49:22.665+0000 7f47f8679700 -1 osd.2 0 waiting for initial osdmap 2023-01-25T06:49:22.674 INFO:tasks.ceph.osd.0.smithi117.stderr:2023-01-25T06:49:22.669+0000 7fe4abb29700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:49:22.674 INFO:tasks.ceph.osd.2.smithi117.stderr:2023-01-25T06:49:22.669+0000 7f47f4130700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:49:22.697 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-01-25T06:49:22.693+0000 7f4599ab1700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-01-25T06:49:24.413 DEBUG:teuthology.orchestra.run.smithi117:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T06:49:24.517 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15d5767700 1 Processor -- start 2023-01-25T06:49:24.519 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15d5767700 1 -- start start 2023-01-25T06:49:24.519 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15d5767700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15d009aa20 0x7f15d009adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:24.519 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15d5767700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d006d9c0 con 0x7f15d009b330 2023-01-25T06:49:24.519 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15d5767700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d0099f00 con 0x7f15d009aa20 2023-01-25T06:49:24.527 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15cf7fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f15d009b330 0x7f15d012f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:41956/0 (socket says 172.21.15.117:41956) 2023-01-25T06:49:24.527 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15cffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15d009aa20 0x7f15d009adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:24.528 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15cf7fe700 1 -- 172.21.15.117:0/4233350772 learned_addr learned my addr 172.21.15.117:0/4233350772 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:24.528 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15ceffd700 1 -- 172.21.15.117:0/4233350772 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904602271 0 0) 0x7f15d006d9c0 con 0x7f15d009b330 2023-01-25T06:49:24.529 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15ceffd700 1 -- 172.21.15.117:0/4233350772 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15b00035d0 con 0x7f15d009b330 2023-01-25T06:49:24.529 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15ceffd700 1 -- 172.21.15.117:0/4233350772 <== mon.0 v1:172.21.15.117:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2092673723 0 0) 0x7f15b00035d0 con 0x7f15d009b330 2023-01-25T06:49:24.530 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15cffff700 1 -- 172.21.15.117:0/4233350772 >> v1:172.21.15.117:6789/0 conn(0x7f15d009b330 legacy=0x7f15d012f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:24.530 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15cffff700 1 -- 172.21.15.117:0/4233350772 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d006d6b0 con 0x7f15d009aa20 2023-01-25T06:49:24.531 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.513+0000 7f15cffff700 1 --2- 172.21.15.117:0/4233350772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 0x7f15d009adf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f15c00097d0 tx=0x7f15c0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecc5589f6d8c652 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:24.531 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15ceffd700 1 -- 172.21.15.117:0/4233350772 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15b80015a0 con 0x7f15d009aa20 2023-01-25T06:49:24.531 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15ceffd700 1 -- 172.21.15.117:0/4233350772 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c0004030 con 0x7f15d009aa20 2023-01-25T06:49:24.531 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15ceffd700 1 -- 172.21.15.117:0/4233350772 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15c00042d0 con 0x7f15d009aa20 2023-01-25T06:49:24.532 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 -- 172.21.15.117:0/4233350772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 msgr2=0x7f15d009adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:24.532 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 --2- 172.21.15.117:0/4233350772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 0x7f15d009adf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f15c00097d0 tx=0x7f15c0009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:24.532 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 -- 172.21.15.117:0/4233350772 shutdown_connections 2023-01-25T06:49:24.532 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 --2- 172.21.15.117:0/4233350772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 0x7f15d009adf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:24.533 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 -- 172.21.15.117:0/4233350772 >> 172.21.15.117:0/4233350772 conn(0x7f15d009cb20 msgr2=0x7f15d009cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:24.533 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 -- 172.21.15.117:0/4233350772 shutdown_connections 2023-01-25T06:49:24.533 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 -- 172.21.15.117:0/4233350772 wait complete. 2023-01-25T06:49:24.533 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 Processor -- start 2023-01-25T06:49:24.533 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 -- start start 2023-01-25T06:49:24.534 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 0x7f15d011c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:24.534 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0071570 con 0x7f15d009aa20 2023-01-25T06:49:24.534 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15cffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 0x7f15d011c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:24.534 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15cffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 0x7f15d011c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48928/0 (socket says 172.21.15.117:48928) 2023-01-25T06:49:24.534 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15cffff700 1 -- 172.21.15.117:0/1615799763 learned_addr learned my addr 172.21.15.117:0/1615799763 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:24.535 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15cffff700 1 -- 172.21.15.117:0/1615799763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c00094b0 con 0x7f15d009aa20 2023-01-25T06:49:24.535 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15cffff700 1 --2- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 0x7f15d011c560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15d0071970 tx=0x7f15c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:24.535 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15ccff9700 1 -- 172.21.15.117:0/1615799763 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15c00106e0 con 0x7f15d009aa20 2023-01-25T06:49:24.535 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.517+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d006cfd0 con 0x7f15d009aa20 2023-01-25T06:49:24.535 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.521+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d006d300 con 0x7f15d009aa20 2023-01-25T06:49:24.536 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.521+0000 7f15ccff9700 1 -- 172.21.15.117:0/1615799763 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c0010840 con 0x7f15d009aa20 2023-01-25T06:49:24.536 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.521+0000 7f15ccff9700 1 -- 172.21.15.117:0/1615799763 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15c0010ae0 con 0x7f15d009aa20 2023-01-25T06:49:24.536 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.521+0000 7f15ccff9700 1 -- 172.21.15.117:0/1615799763 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 3) v1 ==== 41434+0+0 (secure 0 0 0) 0x7f15c000f070 con 0x7f15d009aa20 2023-01-25T06:49:24.536 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.521+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a8005100 con 0x7f15d009aa20 2023-01-25T06:49:24.537 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.521+0000 7f15ccff9700 1 --2- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b4033d00 0x7f15b40361b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:24.537 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.521+0000 7f15ccff9700 1 -- 172.21.15.117:0/1615799763 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2201+0+0 (secure 0 0 0) 0x7f15c0046940 con 0x7f15d009aa20 2023-01-25T06:49:24.537 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.521+0000 7f15cf7fe700 1 --2- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b4033d00 0x7f15b40361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:24.537 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.521+0000 7f15cf7fe700 1 --2- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b4033d00 0x7f15b40361b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f15b8001790 tx=0x7f15b80010e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:24.538 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.529+0000 7f15ccff9700 1 -- 172.21.15.117:0/1615799763 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f15c001a970 con 0x7f15d009aa20 2023-01-25T06:49:24.682 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.677+0000 7f15ccff9700 1 -- 172.21.15.117:0/1615799763 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f15c001cd10 con 0x7f15d009aa20 2023-01-25T06:49:24.698 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.693+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15a80053b0 con 0x7f15d009aa20 2023-01-25T06:49:24.699 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.693+0000 7f15ccff9700 1 -- 172.21.15.117:0/1615799763 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+4772 (secure 0 0 0) 0x7f15c0016070 con 0x7f15d009aa20 2023-01-25T06:49:24.699 INFO:teuthology.misc.health.smithi117.stdout: 2023-01-25T06:49:24.700 INFO:teuthology.misc.health.smithi117.stdout:{"epoch":10,"fsid":"52037823-4da1-437a-aebe-937b909d2d65","created":"2023-01-25T06:49:16.306989+0000","modified":"2023-01-25T06:49:24.672138+0000","last_up_change":"2023-01-25T06:49:23.663021+0000","last_in_change":"2023-01-25T06:49:17.386973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"12f2c557-9775-4b67-a383-f8362a26e4fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":25717},{"type":"v1","addr":"172.21.15.117:6801","nonce":25717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":25717},{"type":"v1","addr":"172.21.15.117:6803","nonce":25717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":25717},{"type":"v1","addr":"172.21.15.117:6807","nonce":25717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":25717},{"type":"v1","addr":"172.21.15.117:6805","nonce":25717}]},"public_addr":"172.21.15.117:6801/25717","cluster_addr":"172.21.15.117:6803/25717","heartbeat_back_addr":"172.21.15.117:6807/25717","heartbeat_front_addr":"172.21.15.117:6805/25717","state":["exists","up"]},{"osd":1,"uuid":"dd866452-4e42-4fdd-8635-12cead61e275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":25718},{"type":"v1","addr":"172.21.15.117:6809","nonce":25718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":25718},{"type":"v1","addr":"172.21.15.117:6811","nonce":25718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":25718},{"type":"v1","addr":"172.21.15.117:6815","nonce":25718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":25718},{"type":"v1","addr":"172.21.15.117:6813","nonce":25718}]},"public_addr":"172.21.15.117:6809/25718","cluster_addr":"172.21.15.117:6811/25718","heartbeat_back_addr":"172.21.15.117:6815/25718","heartbeat_front_addr":"172.21.15.117:6813/25718","state":["exists","up"]},{"osd":2,"uuid":"ede1a6e7-9acf-4606-835e-e3518ce94df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":25720},{"type":"v1","addr":"172.21.15.117:6817","nonce":25720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":25720},{"type":"v1","addr":"172.21.15.117:6819","nonce":25720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":25720},{"type":"v1","addr":"172.21.15.117:6823","nonce":25720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":25720},{"type":"v1","addr":"172.21.15.117:6821","nonce":25720}]},"public_addr":"172.21.15.117:6817/25720","cluster_addr":"172.21.15.117:6819/25720","heartbeat_back_addr":"172.21.15.117:6823/25720","heartbeat_front_addr":"172.21.15.117:6821/25720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.868106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.616222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.917565+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:49:24.702 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b4033d00 msgr2=0x7f15b40361b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:24.702 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 --2- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b4033d00 0x7f15b40361b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f15b8001790 tx=0x7f15b80010e0 comp rx=0 tx=0).stop 2023-01-25T06:49:24.702 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 msgr2=0x7f15d011c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:24.702 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 --2- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 0x7f15d011c560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15d0071970 tx=0x7f15c0004770 comp rx=0 tx=0).stop 2023-01-25T06:49:24.703 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 shutdown_connections 2023-01-25T06:49:24.703 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 --2- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b4033d00 0x7f15b40361b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:24.703 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 --2- 172.21.15.117:0/1615799763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009aa20 0x7f15d011c560 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:24.703 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 >> 172.21.15.117:0/1615799763 conn(0x7f15d009cb20 msgr2=0x7f15d006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:24.703 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 shutdown_connections 2023-01-25T06:49:24.704 INFO:teuthology.misc.health.smithi117.stderr:2023-01-25T06:49:24.697+0000 7f15d5767700 1 -- 172.21.15.117:0/1615799763 wait complete. 2023-01-25T06:49:24.712 DEBUG:teuthology.misc:3 of 3 OSDs are up 2023-01-25T06:49:24.712 INFO:tasks.ceph:Creating RBD pool 2023-01-25T06:49:24.712 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-25T06:49:24.822 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 Processor -- start 2023-01-25T06:49:24.822 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 -- start start 2023-01-25T06:49:24.822 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc02c0985c0 0x7fc02c13b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:24.822 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c099380 con 0x7fc02c09cbb0 2023-01-25T06:49:24.823 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0991a0 con 0x7fc02c0985c0 2023-01-25T06:49:24.823 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc02bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc02c0985c0 0x7fc02c13b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:24.823 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc02bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc02c0985c0 0x7fc02c13b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48930/0 (socket says 172.21.15.117:48930) 2023-01-25T06:49:24.823 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc02bfff700 1 -- 172.21.15.117:0/1651030740 learned_addr learned my addr 172.21.15.117:0/1651030740 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:24.824 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc030fb8700 1 --1- 172.21.15.117:0/1651030740 >> v1:172.21.15.117:6789/0 conn(0x7fc02c09cbb0 0x7fc02c09cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:41972/0 (socket says 172.21.15.117:41972) 2023-01-25T06:49:24.824 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc02bfff700 1 -- 172.21.15.117:0/1651030740 >> v1:172.21.15.117:6789/0 conn(0x7fc02c09cbb0 legacy=0x7fc02c09cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:24.824 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc02bfff700 1 -- 172.21.15.117:0/1651030740 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0975b0 con 0x7fc02c0985c0 2023-01-25T06:49:24.824 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc02bfff700 1 --2- 172.21.15.117:0/1651030740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 0x7fc02c13b4f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009890 tx=0x7fc01c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3510616eb8c9ed6 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:24.824 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc02b7fe700 1 -- 172.21.15.117:0/1651030740 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc01c004160 con 0x7fc02c0985c0 2023-01-25T06:49:24.825 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc02b7fe700 1 -- 172.21.15.117:0/1651030740 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c0042c0 con 0x7fc02c0985c0 2023-01-25T06:49:24.825 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc02b7fe700 1 -- 172.21.15.117:0/1651030740 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc01c004580 con 0x7fc02c0985c0 2023-01-25T06:49:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 -- 172.21.15.117:0/1651030740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 msgr2=0x7fc02c13b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 --2- 172.21.15.117:0/1651030740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 0x7fc02c13b4f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009890 tx=0x7fc01c009ba0 comp rx=0 tx=0).stop 2023-01-25T06:49:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 -- 172.21.15.117:0/1651030740 shutdown_connections 2023-01-25T06:49:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 --2- 172.21.15.117:0/1651030740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 0x7fc02c13b4f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 -- 172.21.15.117:0/1651030740 >> 172.21.15.117:0/1651030740 conn(0x7fc02c09e020 msgr2=0x7fc02c09e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 -- 172.21.15.117:0/1651030740 shutdown_connections 2023-01-25T06:49:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.817+0000 7fc031fba700 1 -- 172.21.15.117:0/1651030740 wait complete. 2023-01-25T06:49:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc031fba700 1 Processor -- start 2023-01-25T06:49:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc031fba700 1 -- start start 2023-01-25T06:49:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc031fba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 0x7fc02c1c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc031fba700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c1c6d20 con 0x7fc02c0985c0 2023-01-25T06:49:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc030fb8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 0x7fc02c1c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc030fb8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 0x7fc02c1c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48940/0 (socket says 172.21.15.117:48940) 2023-01-25T06:49:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc030fb8700 1 -- 172.21.15.117:0/968850637 learned_addr learned my addr 172.21.15.117:0/968850637 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc030fb8700 1 -- 172.21.15.117:0/968850637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c009570 con 0x7fc02c0985c0 2023-01-25T06:49:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc030fb8700 1 --2- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 0x7fc02c1c7220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c480 tx=0x7fc02000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc0297fa700 1 -- 172.21.15.117:0/968850637 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc020007410 con 0x7fc02c0985c0 2023-01-25T06:49:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc0297fa700 1 -- 172.21.15.117:0/968850637 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc020007570 con 0x7fc02c0985c0 2023-01-25T06:49:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c097b80 con 0x7fc02c0985c0 2023-01-25T06:49:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc0297fa700 1 -- 172.21.15.117:0/968850637 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc020005850 con 0x7fc02c0985c0 2023-01-25T06:49:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.821+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c097e90 con 0x7fc02c0985c0 2023-01-25T06:49:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.825+0000 7fc0297fa700 1 -- 172.21.15.117:0/968850637 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7fc020019030 con 0x7fc02c0985c0 2023-01-25T06:49:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.825+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c0a0320 con 0x7fc02c0985c0 2023-01-25T06:49:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.825+0000 7fc0297fa700 1 --2- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fc018033d30 0x7fc0180361e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.825+0000 7fc0297fa700 1 -- 172.21.15.117:0/968850637 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2201+0+0 (secure 0 0 0) 0x7fc0200444d0 con 0x7fc02c0985c0 2023-01-25T06:49:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.825+0000 7fc02bfff700 1 --2- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fc018033d30 0x7fc0180361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.825+0000 7fc02bfff700 1 --2- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fc018033d30 0x7fc0180361e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009fd0 tx=0x7fc01c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.833+0000 7fc0297fa700 1 -- 172.21.15.117:0/968850637 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7fc020019890 con 0x7fc02c0985c0 2023-01-25T06:49:24.996 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:24.993+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fc02c09a100 con 0x7fc02c0985c0 2023-01-25T06:49:25.689 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.685+0000 7fc0297fa700 1 -- 172.21.15.117:0/968850637 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v11) v1 ==== 109+0+0 (secure 0 0 0) 0x7fc020017070 con 0x7fc02c0985c0 2023-01-25T06:49:25.696 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fc018033d30 msgr2=0x7fc0180361e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:25.696 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 --2- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fc018033d30 0x7fc0180361e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009fd0 tx=0x7fc01c008030 comp rx=0 tx=0).stop 2023-01-25T06:49:25.697 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 msgr2=0x7fc02c1c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:25.697 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 --2- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 0x7fc02c1c7220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c480 tx=0x7fc02000c840 comp rx=0 tx=0).stop 2023-01-25T06:49:25.697 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 shutdown_connections 2023-01-25T06:49:25.698 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 --2- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fc018033d30 0x7fc0180361e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:25.698 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 --2- 172.21.15.117:0/968850637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc02c0985c0 0x7fc02c1c7220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:25.698 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 >> 172.21.15.117:0/968850637 conn(0x7fc02c09e020 msgr2=0x7fc02c06cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:25.699 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 shutdown_connections 2023-01-25T06:49:25.699 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.693+0000 7fc031fba700 1 -- 172.21.15.117:0/968850637 wait complete. 2023-01-25T06:49:25.700 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'rbd' created 2023-01-25T06:49:25.710 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-25T06:49:25.828 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b359e700 1 Processor -- start 2023-01-25T06:49:25.829 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b359e700 1 -- start start 2023-01-25T06:49:25.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b359e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58b4065330 0x7f58b412fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:25.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b359e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b412f870 con 0x7f58b41300c0 2023-01-25T06:49:25.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b359e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b4065700 con 0x7f58b4065330 2023-01-25T06:49:25.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b259c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f58b4065330 0x7f58b412fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:25.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b1d9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f58b41300c0 0x7f58b406f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:41978/0 (socket says 172.21.15.117:41978) 2023-01-25T06:49:25.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b1d9b700 1 -- 172.21.15.117:0/2042585343 learned_addr learned my addr 172.21.15.117:0/2042585343 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:25.833 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b259c700 1 -- 172.21.15.117:0/2042585343 >> v1:172.21.15.117:6789/0 conn(0x7f58b41300c0 legacy=0x7f58b406f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:25.833 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b159a700 1 -- 172.21.15.117:0/2042585343 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148806029 0 0) 0x7f58b412f870 con 0x7f58b41300c0 2023-01-25T06:49:25.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b259c700 1 -- 172.21.15.117:0/2042585343 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b409f6e0 con 0x7f58b4065330 2023-01-25T06:49:25.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b259c700 1 --2- 172.21.15.117:0/2042585343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 0x7f58b412fb80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58a40097d0 tx=0x7f58a4009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd1ff0754386529 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:25.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.825+0000 7f58b159a700 1 -- 172.21.15.117:0/2042585343 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58a40041c0 con 0x7f58b4065330 2023-01-25T06:49:25.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b159a700 1 -- 172.21.15.117:0/2042585343 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a4004320 con 0x7f58b4065330 2023-01-25T06:49:25.834 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b159a700 1 -- 172.21.15.117:0/2042585343 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58a40036c0 con 0x7f58b4065330 2023-01-25T06:49:25.835 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 -- 172.21.15.117:0/2042585343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 msgr2=0x7f58b412fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:25.835 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 --2- 172.21.15.117:0/2042585343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 0x7f58b412fb80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58a40097d0 tx=0x7f58a4009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:25.835 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 -- 172.21.15.117:0/2042585343 shutdown_connections 2023-01-25T06:49:25.835 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 --2- 172.21.15.117:0/2042585343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 0x7f58b412fb80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:25.835 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 -- 172.21.15.117:0/2042585343 >> 172.21.15.117:0/2042585343 conn(0x7f58b409d650 msgr2=0x7f58b409da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:25.836 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 -- 172.21.15.117:0/2042585343 shutdown_connections 2023-01-25T06:49:25.836 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 -- 172.21.15.117:0/2042585343 wait complete. 2023-01-25T06:49:25.838 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 Processor -- start 2023-01-25T06:49:25.838 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 -- start start 2023-01-25T06:49:25.838 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 0x7f58b41c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:25.838 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b359e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b412d2e0 con 0x7f58b4065330 2023-01-25T06:49:25.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 0x7f58b41c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:25.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 0x7f58b41c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48948/0 (socket says 172.21.15.117:48948) 2023-01-25T06:49:25.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.829+0000 7f58b259c700 1 -- 172.21.15.117:0/3548317802 learned_addr learned my addr 172.21.15.117:0/3548317802 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:25.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58b259c700 1 -- 172.21.15.117:0/3548317802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40094b0 con 0x7f58b4065330 2023-01-25T06:49:25.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58b259c700 1 --2- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 0x7f58b41c3f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009b20 tx=0x7f58a4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:25.840 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58aaffd700 1 -- 172.21.15.117:0/3548317802 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58a4006ea0 con 0x7f58b4065330 2023-01-25T06:49:25.840 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58aaffd700 1 -- 172.21.15.117:0/3548317802 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a4011630 con 0x7f58b4065330 2023-01-25T06:49:25.840 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b41304e0 con 0x7f58b4065330 2023-01-25T06:49:25.840 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58aaffd700 1 -- 172.21.15.117:0/3548317802 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58a40118d0 con 0x7f58b4065330 2023-01-25T06:49:25.840 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b40a0250 con 0x7f58b4065330 2023-01-25T06:49:25.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b409ea80 con 0x7f58b4065330 2023-01-25T06:49:25.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58aaffd700 1 -- 172.21.15.117:0/3548317802 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f58a402d070 con 0x7f58b4065330 2023-01-25T06:49:25.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58aaffd700 1 --2- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5894033d90 0x7f5894036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:25.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58aaffd700 1 -- 172.21.15.117:0/3548317802 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2543+0+0 (secure 0 0 0) 0x7f58a4046100 con 0x7f58b4065330 2023-01-25T06:49:25.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58b1d9b700 1 --2- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5894033d90 0x7f5894036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:25.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.833+0000 7f58b1d9b700 1 --2- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5894033d90 0x7f5894036240 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f589c000c00 tx=0x7f589c0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:25.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:25.845+0000 7f58aaffd700 1 -- 172.21.15.117:0/3548317802 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f58a4024770 con 0x7f58b4065330 2023-01-25T06:49:26.013 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.009+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f58b404ebb0 con 0x7f58b4065330 2023-01-25T06:49:26.700 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.697+0000 7f58aaffd700 1 -- 172.21.15.117:0/3548317802 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v12) v1 ==== 173+0+0 (secure 0 0 0) 0x7f58a4016020 con 0x7f58b4065330 2023-01-25T06:49:26.706 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5894033d90 msgr2=0x7f5894036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:26.707 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 --2- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5894033d90 0x7f5894036240 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f589c000c00 tx=0x7f589c0079c0 comp rx=0 tx=0).stop 2023-01-25T06:49:26.707 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 msgr2=0x7f58b41c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:26.707 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 --2- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 0x7f58b41c3f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009b20 tx=0x7f58a4004770 comp rx=0 tx=0).stop 2023-01-25T06:49:26.707 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 shutdown_connections 2023-01-25T06:49:26.708 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 --2- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5894033d90 0x7f5894036240 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:26.708 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 --2- 172.21.15.117:0/3548317802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58b4065330 0x7f58b41c3f80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:26.708 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 >> 172.21.15.117:0/3548317802 conn(0x7f58b409d650 msgr2=0x7f58b41b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:26.708 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 shutdown_connections 2023-01-25T06:49:26.709 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.701+0000 7f58b359e700 1 -- 172.21.15.117:0/3548317802 wait complete. 2023-01-25T06:49:26.709 INFO:teuthology.orchestra.run.smithi117.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-25T06:49:26.723 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-25T06:49:26.723 INFO:tasks.ceph.mds.a:Restarting daemon 2023-01-25T06:49:26.724 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-01-25T06:49:26.726 INFO:tasks.ceph.mds.a:Started 2023-01-25T06:49:26.726 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-01-25T06:49:26.727 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T06:49:26.796 INFO:tasks.ceph.mds.a.smithi117.stdout:starting mds.a at 2023-01-25T06:49:26.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.833+0000 7fb18afa3700 1 Processor -- start 2023-01-25T06:49:26.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.833+0000 7fb18afa3700 1 -- start start 2023-01-25T06:49:26.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb18406e780 0x7fb184070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:26.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1840a0460 con 0x7fb18406f850 2023-01-25T06:49:26.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18406f540 con 0x7fb18406e780 2023-01-25T06:49:26.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb1897a0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb18406e780 0x7fb184070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:26.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb1897a0700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb18406e780 0x7fb184070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48978/0 (socket says 172.21.15.117:48978) 2023-01-25T06:49:26.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb189fa1700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb18406f850 0x7fb18406fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:41988/0 (socket says 172.21.15.117:41988) 2023-01-25T06:49:26.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb1897a0700 1 -- 172.21.15.117:0/1712922843 learned_addr learned my addr 172.21.15.117:0/1712922843 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:26.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb1897a0700 1 -- 172.21.15.117:0/1712922843 >> v1:172.21.15.117:6789/0 conn(0x7fb18406f850 legacy=0x7fb18406fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:26.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb1897a0700 1 -- 172.21.15.117:0/1712922843 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1840713e0 con 0x7fb18406e780 2023-01-25T06:49:26.843 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb1897a0700 1 --2- 172.21.15.117:0/1712922843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 0x7fb184070330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb174009890 tx=0x7fb174009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c4eae9fb2408c2 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:26.843 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb188f9f700 1 -- 172.21.15.117:0/1712922843 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb174004160 con 0x7fb18406e780 2023-01-25T06:49:26.843 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb188f9f700 1 -- 172.21.15.117:0/1712922843 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1740042c0 con 0x7fb18406e780 2023-01-25T06:49:26.843 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb188f9f700 1 -- 172.21.15.117:0/1712922843 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb174004580 con 0x7fb18406e780 2023-01-25T06:49:26.845 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 -- 172.21.15.117:0/1712922843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 msgr2=0x7fb184070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:26.846 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 --2- 172.21.15.117:0/1712922843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 0x7fb184070330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb174009890 tx=0x7fb174009ba0 comp rx=0 tx=0).stop 2023-01-25T06:49:26.846 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 -- 172.21.15.117:0/1712922843 shutdown_connections 2023-01-25T06:49:26.846 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 --2- 172.21.15.117:0/1712922843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 0x7fb184070330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:26.846 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 -- 172.21.15.117:0/1712922843 >> 172.21.15.117:0/1712922843 conn(0x7fb18409d470 msgr2=0x7fb18409d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:26.846 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 -- 172.21.15.117:0/1712922843 shutdown_connections 2023-01-25T06:49:26.847 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.837+0000 7fb18afa3700 1 -- 172.21.15.117:0/1712922843 wait complete. 2023-01-25T06:49:26.847 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb18afa3700 1 Processor -- start 2023-01-25T06:49:26.847 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb18afa3700 1 -- start start 2023-01-25T06:49:26.847 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb18afa3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 0x7fb18411bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:26.847 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb18afa3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb184134240 con 0x7fb18406e780 2023-01-25T06:49:26.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb189fa1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 0x7fb18411bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:26.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb189fa1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 0x7fb18411bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48980/0 (socket says 172.21.15.117:48980) 2023-01-25T06:49:26.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb189fa1700 1 -- 172.21.15.117:0/2462337799 learned_addr learned my addr 172.21.15.117:0/2462337799 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:26.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb189fa1700 1 -- 172.21.15.117:0/2462337799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174009570 con 0x7fb18406e780 2023-01-25T06:49:26.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb189fa1700 1 --2- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 0x7fb18411bc30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb18000e8c0 tx=0x7fb18000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:26.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb17a7fc700 1 -- 172.21.15.117:0/2462337799 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1800049c0 con 0x7fb18406e780 2023-01-25T06:49:26.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb17a7fc700 1 -- 172.21.15.117:0/2462337799 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb180004b20 con 0x7fb18406e780 2023-01-25T06:49:26.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18406f0f0 con 0x7fb18406e780 2023-01-25T06:49:26.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb17a7fc700 1 -- 172.21.15.117:0/2462337799 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb180004dc0 con 0x7fb18406e780 2023-01-25T06:49:26.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb184070870 con 0x7fb18406e780 2023-01-25T06:49:26.850 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb17a7fc700 1 -- 172.21.15.117:0/2462337799 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7fb180021070 con 0x7fb18406e780 2023-01-25T06:49:26.851 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.841+0000 7fb17a7fc700 1 --2- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb168033d90 0x7fb168036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:26.851 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.845+0000 7fb17a7fc700 1 -- 172.21.15.117:0/2462337799 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2554+0+0 (secure 0 0 0) 0x7fb180047370 con 0x7fb18406e780 2023-01-25T06:49:26.851 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.845+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb184134010 con 0x7fb18406e780 2023-01-25T06:49:26.851 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.845+0000 7fb1897a0700 1 --2- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb168033d90 0x7fb168036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:26.851 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.845+0000 7fb1897a0700 1 --2- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb168033d90 0x7fb168036240 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb174009890 tx=0x7fb174008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:26.854 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:26.849+0000 7fb17a7fc700 1 -- 172.21.15.117:0/2462337799 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7fb180021300 con 0x7fb18406e780 2023-01-25T06:49:27.014 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.009+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb184070c80 con 0x7fb18406e780 2023-01-25T06:49:27.015 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb17a7fc700 1 -- 172.21.15.117:0/2462337799 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+6303 (secure 0 0 0) 0x7fb180028030 con 0x7fb18406e780 2023-01-25T06:49:27.016 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:27.016 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":12,"fsid":"52037823-4da1-437a-aebe-937b909d2d65","created":"2023-01-25T06:49:16.306989+0000","modified":"2023-01-25T06:49:26.686074+0000","last_up_change":"2023-01-25T06:49:23.663021+0000","last_in_change":"2023-01-25T06:49:17.386973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-25T06:49:25.037352+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12f2c557-9775-4b67-a383-f8362a26e4fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":25717},{"type":"v1","addr":"172.21.15.117:6801","nonce":25717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":25717},{"type":"v1","addr":"172.21.15.117:6803","nonce":25717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":25717},{"type":"v1","addr":"172.21.15.117:6807","nonce":25717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":25717},{"type":"v1","addr":"172.21.15.117:6805","nonce":25717}]},"public_addr":"172.21.15.117:6801/25717","cluster_addr":"172.21.15.117:6803/25717","heartbeat_back_addr":"172.21.15.117:6807/25717","heartbeat_front_addr":"172.21.15.117:6805/25717","state":["exists","up"]},{"osd":1,"uuid":"dd866452-4e42-4fdd-8635-12cead61e275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":25718},{"type":"v1","addr":"172.21.15.117:6809","nonce":25718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":25718},{"type":"v1","addr":"172.21.15.117:6811","nonce":25718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":25718},{"type":"v1","addr":"172.21.15.117:6815","nonce":25718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":25718},{"type":"v1","addr":"172.21.15.117:6813","nonce":25718}]},"public_addr":"172.21.15.117:6809/25718","cluster_addr":"172.21.15.117:6811/25718","heartbeat_back_addr":"172.21.15.117:6815/25718","heartbeat_front_addr":"172.21.15.117:6813/25718","state":["exists","up"]},{"osd":2,"uuid":"ede1a6e7-9acf-4606-835e-e3518ce94df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":25720},{"type":"v1","addr":"172.21.15.117:6817","nonce":25720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":25720},{"type":"v1","addr":"172.21.15.117:6819","nonce":25720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":25720},{"type":"v1","addr":"172.21.15.117:6823","nonce":25720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":25720},{"type":"v1","addr":"172.21.15.117:6821","nonce":25720}]},"public_addr":"172.21.15.117:6817/25720","cluster_addr":"172.21.15.117:6819/25720","heartbeat_back_addr":"172.21.15.117:6823/25720","heartbeat_front_addr":"172.21.15.117:6821/25720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.868106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.616222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.917565+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:49:27.018 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb168033d90 msgr2=0x7fb168036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.018 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 --2- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb168033d90 0x7fb168036240 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb174009890 tx=0x7fb174008030 comp rx=0 tx=0).stop 2023-01-25T06:49:27.018 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 msgr2=0x7fb18411bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.019 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 --2- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 0x7fb18411bc30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb18000e8c0 tx=0x7fb18000ec80 comp rx=0 tx=0).stop 2023-01-25T06:49:27.019 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 shutdown_connections 2023-01-25T06:49:27.019 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 --2- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb168033d90 0x7fb168036240 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.019 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 --2- 172.21.15.117:0/2462337799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb18406e780 0x7fb18411bc30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.019 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 >> 172.21.15.117:0/2462337799 conn(0x7fb18409d470 msgr2=0x7fb184065330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:27.020 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 shutdown_connections 2023-01-25T06:49:27.020 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.013+0000 7fb18afa3700 1 -- 172.21.15.117:0/2462337799 wait complete. 2023-01-25T06:49:27.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-01-25T06:49:25.037352+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-01-25T06:49:27.031 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-01-25T06:49:27.151 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 Processor -- start 2023-01-25T06:49:27.152 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 -- start start 2023-01-25T06:49:27.153 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15d012f060 0x7f15d0136c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:27.153 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d009ad10 con 0x7f15d009a080 2023-01-25T06:49:27.154 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d009a450 con 0x7f15d012f060 2023-01-25T06:49:27.154 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15ceffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15d012f060 0x7f15d0136c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:27.154 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15cf7fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f15d009a080 0x7f15d012e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:42004/0 (socket says 172.21.15.117:42004) 2023-01-25T06:49:27.154 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15ceffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15d012f060 0x7f15d0136c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48982/0 (socket says 172.21.15.117:48982) 2023-01-25T06:49:27.155 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15cf7fe700 1 -- 172.21.15.117:0/794446658 learned_addr learned my addr 172.21.15.117:0/794446658 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:27.155 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15ceffd700 1 -- 172.21.15.117:0/794446658 >> v1:172.21.15.117:6789/0 conn(0x7f15d009a080 legacy=0x7f15d012e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.155 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15ce7fc700 1 -- 172.21.15.117:0/794446658 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268969166 0 0) 0x7f15d009ad10 con 0x7f15d009a080 2023-01-25T06:49:27.155 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15ceffd700 1 -- 172.21.15.117:0/794446658 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d009a900 con 0x7f15d012f060 2023-01-25T06:49:27.157 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15ceffd700 1 --2- 172.21.15.117:0/794446658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d012f060 0x7f15d0136c30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15b80097c0 tx=0x7f15b8009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506832028fb7154c server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:27.157 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15ce7fc700 1 -- 172.21.15.117:0/794446658 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15b8005d80 con 0x7f15d012f060 2023-01-25T06:49:27.157 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15ce7fc700 1 -- 172.21.15.117:0/794446658 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b8004030 con 0x7f15d012f060 2023-01-25T06:49:27.157 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15ce7fc700 1 -- 172.21.15.117:0/794446658 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15b80042f0 con 0x7f15d012f060 2023-01-25T06:49:27.159 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 -- 172.21.15.117:0/794446658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d012f060 msgr2=0x7f15d0136c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.159 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 --2- 172.21.15.117:0/794446658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d012f060 0x7f15d0136c30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15b80097c0 tx=0x7f15b8009ad0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.160 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 -- 172.21.15.117:0/794446658 shutdown_connections 2023-01-25T06:49:27.160 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 --2- 172.21.15.117:0/794446658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d012f060 0x7f15d0136c30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.160 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 -- 172.21.15.117:0/794446658 >> 172.21.15.117:0/794446658 conn(0x7f15d00698e0 msgr2=0x7f15d0069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:27.160 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 -- 172.21.15.117:0/794446658 shutdown_connections 2023-01-25T06:49:27.161 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.149+0000 7f15d4e1d700 1 -- 172.21.15.117:0/794446658 wait complete. 2023-01-25T06:49:27.161 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15d4e1d700 1 Processor -- start 2023-01-25T06:49:27.161 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15d4e1d700 1 -- start start 2023-01-25T06:49:27.162 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15d4e1d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009a080 0x7f15d01c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:27.162 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15d4e1d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d009d280 con 0x7f15d009a080 2023-01-25T06:49:27.162 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15cf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009a080 0x7f15d01c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:27.162 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15cf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009a080 0x7f15d01c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48990/0 (socket says 172.21.15.117:48990) 2023-01-25T06:49:27.163 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15cf7fe700 1 -- 172.21.15.117:0/3116482903 learned_addr learned my addr 172.21.15.117:0/3116482903 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:27.163 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15cf7fe700 1 -- 172.21.15.117:0/3116482903 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8009450 con 0x7f15d009a080 2023-01-25T06:49:27.163 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15cf7fe700 1 --2- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009a080 0x7f15d01c5f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c3d0 tx=0x7f15c000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:27.163 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15c7fff700 1 -- 172.21.15.117:0/3116482903 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15c0004aa0 con 0x7f15d009a080 2023-01-25T06:49:27.163 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15c7fff700 1 -- 172.21.15.117:0/3116482903 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c0004c00 con 0x7f15d009a080 2023-01-25T06:49:27.164 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15c7fff700 1 -- 172.21.15.117:0/3116482903 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15c00070d0 con 0x7f15d009a080 2023-01-25T06:49:27.164 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d012e2c0 con 0x7f15d009a080 2023-01-25T06:49:27.165 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.153+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d012e640 con 0x7f15d009a080 2023-01-25T06:49:27.165 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.157+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d009af10 con 0x7f15d009a080 2023-01-25T06:49:27.166 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.157+0000 7f15c7fff700 1 -- 172.21.15.117:0/3116482903 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f15c001a020 con 0x7f15d009a080 2023-01-25T06:49:27.166 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.157+0000 7f15c7fff700 1 --2- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b0033d90 0x7f15b0036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:27.166 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.157+0000 7f15c7fff700 1 -- 172.21.15.117:0/3116482903 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2554+0+0 (secure 0 0 0) 0x7f15c0044d30 con 0x7f15d009a080 2023-01-25T06:49:27.166 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.157+0000 7f15ceffd700 1 --2- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b0033d90 0x7f15b0036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:27.166 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.157+0000 7f15ceffd700 1 --2- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b0033d90 0x7f15b0036240 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f15d012f430 tx=0x7f15b8008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:27.167 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.161+0000 7f15c7fff700 1 -- 172.21.15.117:0/3116482903 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f15c001bb70 con 0x7f15d009a080 2023-01-25T06:49:27.326 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.321+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f15d004ebb0 con 0x7f15d009a080 2023-01-25T06:49:27.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.321+0000 7f15c7fff700 1 -- 172.21.15.117:0/3116482903 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v12) v1 ==== 92+0+10 (secure 0 0 0) 0x7f15c001b340 con 0x7f15d009a080 2023-01-25T06:49:27.327 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2023-01-25T06:49:27.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b0033d90 msgr2=0x7f15b0036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 --2- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b0033d90 0x7f15b0036240 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f15d012f430 tx=0x7f15b8008070 comp rx=0 tx=0).stop 2023-01-25T06:49:27.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009a080 msgr2=0x7f15d01c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 --2- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009a080 0x7f15d01c5f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c3d0 tx=0x7f15c000c790 comp rx=0 tx=0).stop 2023-01-25T06:49:27.332 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 shutdown_connections 2023-01-25T06:49:27.332 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 --2- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f15b0033d90 0x7f15b0036240 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.333 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 --2- 172.21.15.117:0/3116482903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d009a080 0x7f15d01c5f00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.333 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 >> 172.21.15.117:0/3116482903 conn(0x7f15d00698e0 msgr2=0x7f15d01b9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:27.333 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 shutdown_connections 2023-01-25T06:49:27.333 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.325+0000 7f15d4e1d700 1 -- 172.21.15.117:0/3116482903 wait complete. 2023-01-25T06:49:27.347 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-01-25T06:49:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe168de6700 1 Processor -- start 2023-01-25T06:49:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe168de6700 1 -- start start 2023-01-25T06:49:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe168de6700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe16406a3c0 0x7fe16413b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe168de6700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe164099780 con 0x7fe1640698e0 2023-01-25T06:49:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe168de6700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe164099270 con 0x7fe16406a3c0 2023-01-25T06:49:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe162ffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe16406a3c0 0x7fe16413b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe162ffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe16406a3c0 0x7fe16413b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49000/0 (socket says 172.21.15.117:49000) 2023-01-25T06:49:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe162ffd700 1 -- 172.21.15.117:0/27755258 learned_addr learned my addr 172.21.15.117:0/27755258 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe162ffd700 1 -- 172.21.15.117:0/27755258 >> v1:172.21.15.117:6789/0 conn(0x7fe1640698e0 legacy=0x7fe164069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe162ffd700 1 -- 172.21.15.117:0/27755258 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16409d530 con 0x7fe16406a3c0 2023-01-25T06:49:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe162ffd700 1 --2- 172.21.15.117:0/27755258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe16406a3c0 0x7fe16413b330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe154009890 tx=0x7fe154009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7288fc1534999c8 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:27.461 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe1627fc700 1 -- 172.21.15.117:0/27755258 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe154004160 con 0x7fe16406a3c0 2023-01-25T06:49:27.461 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe1627fc700 1 -- 172.21.15.117:0/27755258 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1540042c0 con 0x7fe16406a3c0 2023-01-25T06:49:27.461 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.453+0000 7fe1627fc700 1 -- 172.21.15.117:0/27755258 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe154004580 con 0x7fe16406a3c0 2023-01-25T06:49:27.463 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 -- 172.21.15.117:0/27755258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe16406a3c0 msgr2=0x7fe16413b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 --2- 172.21.15.117:0/27755258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe16406a3c0 0x7fe16413b330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe154009890 tx=0x7fe154009ba0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 -- 172.21.15.117:0/27755258 shutdown_connections 2023-01-25T06:49:27.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 --2- 172.21.15.117:0/27755258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe16406a3c0 0x7fe16413b330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 -- 172.21.15.117:0/27755258 >> 172.21.15.117:0/27755258 conn(0x7fe16409e020 msgr2=0x7fe16409e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:27.465 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 -- 172.21.15.117:0/27755258 shutdown_connections 2023-01-25T06:49:27.465 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 -- 172.21.15.117:0/27755258 wait complete. 2023-01-25T06:49:27.465 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 Processor -- start 2023-01-25T06:49:27.465 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 -- start start 2023-01-25T06:49:27.466 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1640698e0 0x7fe1641aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:27.466 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe168de6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1641ac1a0 con 0x7fe1640698e0 2023-01-25T06:49:27.466 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe1637fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1640698e0 0x7fe1641aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:27.466 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe1637fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1640698e0 0x7fe1641aba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49014/0 (socket says 172.21.15.117:49014) 2023-01-25T06:49:27.466 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe1637fe700 1 -- 172.21.15.117:0/1719382007 learned_addr learned my addr 172.21.15.117:0/1719382007 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:27.467 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.457+0000 7fe1637fe700 1 -- 172.21.15.117:0/1719382007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe154009570 con 0x7fe1640698e0 2023-01-25T06:49:27.467 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe1637fe700 1 --2- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1640698e0 0x7fe1641aba50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe15800c380 tx=0x7fe15800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:27.467 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe14bfff700 1 -- 172.21.15.117:0/1719382007 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1580052e0 con 0x7fe1640698e0 2023-01-25T06:49:27.467 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe14bfff700 1 -- 172.21.15.117:0/1719382007 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158005440 con 0x7fe1640698e0 2023-01-25T06:49:27.467 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe164097fe0 con 0x7fe1640698e0 2023-01-25T06:49:27.468 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe14bfff700 1 -- 172.21.15.117:0/1719382007 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1580036a0 con 0x7fe1640698e0 2023-01-25T06:49:27.468 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16409d8e0 con 0x7fe1640698e0 2023-01-25T06:49:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe164099470 con 0x7fe1640698e0 2023-01-25T06:49:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe14bfff700 1 -- 172.21.15.117:0/1719382007 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7fe158019030 con 0x7fe1640698e0 2023-01-25T06:49:27.469 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe14bfff700 1 --2- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe150033d30 0x7fe1500361e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:27.470 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe14bfff700 1 -- 172.21.15.117:0/1719382007 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2554+0+0 (secure 0 0 0) 0x7fe1580444b0 con 0x7fe1640698e0 2023-01-25T06:49:27.470 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe162ffd700 1 --2- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe150033d30 0x7fe1500361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:27.470 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.461+0000 7fe162ffd700 1 --2- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe150033d30 0x7fe1500361e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe154009fd0 tx=0x7fe154008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:27.473 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.469+0000 7fe14bfff700 1 -- 172.21.15.117:0/1719382007 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7fe1580198b0 con 0x7fe1640698e0 2023-01-25T06:49:27.667 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.661+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe164099f40 con 0x7fe1640698e0 2023-01-25T06:49:27.668 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe14bfff700 1 -- 172.21.15.117:0/1719382007 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7fe158017070 con 0x7fe1640698e0 2023-01-25T06:49:27.668 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:27.668 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6827/2542771210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2542771210},{"type":"v1","addr":"172.21.15.117:6827","nonce":2542771210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-01-25T06:49:27.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe150033d30 msgr2=0x7fe1500361e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 --2- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe150033d30 0x7fe1500361e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe154009fd0 tx=0x7fe154008030 comp rx=0 tx=0).stop 2023-01-25T06:49:27.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1640698e0 msgr2=0x7fe1641aba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 --2- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1640698e0 0x7fe1641aba50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe15800c380 tx=0x7fe15800c740 comp rx=0 tx=0).stop 2023-01-25T06:49:27.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 shutdown_connections 2023-01-25T06:49:27.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 --2- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe150033d30 0x7fe1500361e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.672 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 --2- 172.21.15.117:0/1719382007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1640698e0 0x7fe1641aba50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.672 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 >> 172.21.15.117:0/1719382007 conn(0x7fe16409e020 msgr2=0x7fe1641be040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:27.672 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 shutdown_connections 2023-01-25T06:49:27.672 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.665+0000 7fe168de6700 1 -- 172.21.15.117:0/1719382007 wait complete. 2023-01-25T06:49:27.673 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 2 2023-01-25T06:49:27.683 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T06:49:27.795 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.789+0000 7f77e592c700 1 Processor -- start 2023-01-25T06:49:27.795 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.789+0000 7f77e592c700 1 -- start start 2023-01-25T06:49:27.795 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.789+0000 7f77e592c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f77e01316c0 0x7f77e0136a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:27.795 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.789+0000 7f77e592c700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e0098d80 con 0x7f77e0099e60 2023-01-25T06:49:27.796 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.789+0000 7f77e592c700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e009a230 con 0x7f77e01316c0 2023-01-25T06:49:27.797 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.789+0000 7f77e492a700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f77e0099e60 0x7f77e0131e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:42024/0 (socket says 172.21.15.117:42024) 2023-01-25T06:49:27.797 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.789+0000 7f77d7fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f77e01316c0 0x7f77e0136a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:27.797 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.789+0000 7f77e492a700 1 -- 172.21.15.117:0/2003608453 learned_addr learned my addr 172.21.15.117:0/2003608453 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:27.798 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77d7fff700 1 -- 172.21.15.117:0/2003608453 >> v1:172.21.15.117:6789/0 conn(0x7f77e0099e60 legacy=0x7f77e0131e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.798 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77d7fff700 1 -- 172.21.15.117:0/2003608453 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e009ba70 con 0x7f77e01316c0 2023-01-25T06:49:27.798 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77d7fff700 1 --2- 172.21.15.117:0/2003608453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e01316c0 0x7f77e0136a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77d00097d0 tx=0x7f77d0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85256b51a2be0e2d server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:27.798 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77d77fe700 1 -- 172.21.15.117:0/2003608453 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d00041c0 con 0x7f77e01316c0 2023-01-25T06:49:27.798 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77d77fe700 1 -- 172.21.15.117:0/2003608453 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d0004320 con 0x7f77e01316c0 2023-01-25T06:49:27.799 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77d77fe700 1 -- 172.21.15.117:0/2003608453 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d00036c0 con 0x7f77e01316c0 2023-01-25T06:49:27.800 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77e592c700 1 -- 172.21.15.117:0/2003608453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e01316c0 msgr2=0x7f77e0136a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.800 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77e592c700 1 --2- 172.21.15.117:0/2003608453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e01316c0 0x7f77e0136a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f77d00097d0 tx=0x7f77d0009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.800 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77e592c700 1 -- 172.21.15.117:0/2003608453 shutdown_connections 2023-01-25T06:49:27.800 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77e592c700 1 --2- 172.21.15.117:0/2003608453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e01316c0 0x7f77e0136a40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.801 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77e592c700 1 -- 172.21.15.117:0/2003608453 >> 172.21.15.117:0/2003608453 conn(0x7f77e00a5170 msgr2=0x7f77e00698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:27.801 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77e592c700 1 -- 172.21.15.117:0/2003608453 shutdown_connections 2023-01-25T06:49:27.801 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.793+0000 7f77e592c700 1 -- 172.21.15.117:0/2003608453 wait complete. 2023-01-25T06:49:27.801 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e592c700 1 Processor -- start 2023-01-25T06:49:27.801 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e592c700 1 -- start start 2023-01-25T06:49:27.803 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e592c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e0099e60 0x7f77e009cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:27.804 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e592c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00a1120 con 0x7f77e0099e60 2023-01-25T06:49:27.804 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e492a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e0099e60 0x7f77e009cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:27.804 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e492a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e0099e60 0x7f77e009cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49032/0 (socket says 172.21.15.117:49032) 2023-01-25T06:49:27.804 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e492a700 1 -- 172.21.15.117:0/794780583 learned_addr learned my addr 172.21.15.117:0/794780583 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e492a700 1 -- 172.21.15.117:0/794780583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00094b0 con 0x7f77e0099e60 2023-01-25T06:49:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e492a700 1 --2- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e0099e60 0x7f77e009cd60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00c480 tx=0x7f77dc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77d57fa700 1 -- 172.21.15.117:0/794780583 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77dc005cc0 con 0x7f77e0099e60 2023-01-25T06:49:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77d57fa700 1 -- 172.21.15.117:0/794780583 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc005e20 con 0x7f77e0099e60 2023-01-25T06:49:27.805 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e009a650 con 0x7f77e0099e60 2023-01-25T06:49:27.806 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77d57fa700 1 -- 172.21.15.117:0/794780583 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77dc0049c0 con 0x7f77e0099e60 2023-01-25T06:49:27.806 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.797+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e0131b60 con 0x7f77e0099e60 2023-01-25T06:49:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.801+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e0132580 con 0x7f77e0099e60 2023-01-25T06:49:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.801+0000 7f77d57fa700 1 -- 172.21.15.117:0/794780583 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f77dc010070 con 0x7f77e0099e60 2023-01-25T06:49:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.801+0000 7f77d57fa700 1 --2- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f77cc033d90 0x7f77cc036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:27.807 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.801+0000 7f77d57fa700 1 -- 172.21.15.117:0/794780583 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2554+0+0 (secure 0 0 0) 0x7f77dc044960 con 0x7f77e0099e60 2023-01-25T06:49:27.808 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.801+0000 7f77d7fff700 1 --2- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f77cc033d90 0x7f77cc036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:27.808 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.801+0000 7f77d7fff700 1 --2- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f77cc033d90 0x7f77cc036240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77d00097d0 tx=0x7f77d0010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:27.811 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.805+0000 7f77d57fa700 1 -- 172.21.15.117:0/794780583 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f77dc011760 con 0x7f77e0099e60 2023-01-25T06:49:27.973 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.969+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77e00a16f0 con 0x7f77e0099e60 2023-01-25T06:49:27.974 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.969+0000 7f77d57fa700 1 -- 172.21.15.117:0/794780583 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+6290 (secure 0 0 0) 0x7f77e00a16f0 con 0x7f77e0099e60 2023-01-25T06:49:27.975 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:27.975 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":13,"fsid":"52037823-4da1-437a-aebe-937b909d2d65","created":"2023-01-25T06:49:16.306989+0000","modified":"2023-01-25T06:49:27.695568+0000","last_up_change":"2023-01-25T06:49:23.663021+0000","last_in_change":"2023-01-25T06:49:17.386973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-25T06:49:25.037352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"12f2c557-9775-4b67-a383-f8362a26e4fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":25717},{"type":"v1","addr":"172.21.15.117:6801","nonce":25717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":25717},{"type":"v1","addr":"172.21.15.117:6803","nonce":25717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":25717},{"type":"v1","addr":"172.21.15.117:6807","nonce":25717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":25717},{"type":"v1","addr":"172.21.15.117:6805","nonce":25717}]},"public_addr":"172.21.15.117:6801/25717","cluster_addr":"172.21.15.117:6803/25717","heartbeat_back_addr":"172.21.15.117:6807/25717","heartbeat_front_addr":"172.21.15.117:6805/25717","state":["exists","up"]},{"osd":1,"uuid":"dd866452-4e42-4fdd-8635-12cead61e275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":25718},{"type":"v1","addr":"172.21.15.117:6809","nonce":25718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":25718},{"type":"v1","addr":"172.21.15.117:6811","nonce":25718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":25718},{"type":"v1","addr":"172.21.15.117:6815","nonce":25718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":25718},{"type":"v1","addr":"172.21.15.117:6813","nonce":25718}]},"public_addr":"172.21.15.117:6809/25718","cluster_addr":"172.21.15.117:6811/25718","heartbeat_back_addr":"172.21.15.117:6815/25718","heartbeat_front_addr":"172.21.15.117:6813/25718","state":["exists","up"]},{"osd":2,"uuid":"ede1a6e7-9acf-4606-835e-e3518ce94df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":25720},{"type":"v1","addr":"172.21.15.117:6817","nonce":25720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":25720},{"type":"v1","addr":"172.21.15.117:6819","nonce":25720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":25720},{"type":"v1","addr":"172.21.15.117:6823","nonce":25720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":25720},{"type":"v1","addr":"172.21.15.117:6821","nonce":25720}]},"public_addr":"172.21.15.117:6817/25720","cluster_addr":"172.21.15.117:6819/25720","heartbeat_back_addr":"172.21.15.117:6823/25720","heartbeat_front_addr":"172.21.15.117:6821/25720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.868106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.616222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.917565+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:49:27.977 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f77cc033d90 msgr2=0x7f77cc036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.977 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 --2- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f77cc033d90 0x7f77cc036240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77d00097d0 tx=0x7f77d0010000 comp rx=0 tx=0).stop 2023-01-25T06:49:27.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e0099e60 msgr2=0x7f77e009cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:27.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 --2- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e0099e60 0x7f77e009cd60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00c480 tx=0x7f77dc00c840 comp rx=0 tx=0).stop 2023-01-25T06:49:27.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 shutdown_connections 2023-01-25T06:49:27.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 --2- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f77cc033d90 0x7f77cc036240 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 --2- 172.21.15.117:0/794780583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77e0099e60 0x7f77e009cd60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:27.979 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 >> 172.21.15.117:0/794780583 conn(0x7f77e00a5170 msgr2=0x7f77e01b9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:27.979 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 shutdown_connections 2023-01-25T06:49:27.979 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:27.973+0000 7f77e592c700 1 -- 172.21.15.117:0/794780583 wait complete. 2023-01-25T06:49:27.996 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-01-25T06:49:25.037352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-01-25T06:49:27.996 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-01-25T06:49:28.116 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 Processor -- start 2023-01-25T06:49:28.117 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 -- start start 2023-01-25T06:49:28.118 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5d3812f190 0x7f5d38136d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:28.118 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d38098700 con 0x7f5d38096e80 2023-01-25T06:49:28.119 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d38097b10 con 0x7f5d3812f190 2023-01-25T06:49:28.119 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3ccec700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5d3812f190 0x7f5d38136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:28.119 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3ccec700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5d3812f190 0x7f5d38136d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49046/0 (socket says 172.21.15.117:49046) 2023-01-25T06:49:28.119 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3ccec700 1 -- 172.21.15.117:0/440840611 learned_addr learned my addr 172.21.15.117:0/440840611 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:28.119 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3d4ed700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f5d38096e80 0x7f5d3812ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:42032/0 (socket says 172.21.15.117:42032) 2023-01-25T06:49:28.120 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3ccec700 1 -- 172.21.15.117:0/440840611 >> v1:172.21.15.117:6789/0 conn(0x7f5d38096e80 legacy=0x7f5d3812ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.120 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3ccec700 1 -- 172.21.15.117:0/440840611 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3812e770 con 0x7f5d3812f190 2023-01-25T06:49:28.120 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3ccec700 1 --2- 172.21.15.117:0/440840611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3812f190 0x7f5d38136d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009890 tx=0x7f5d34009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2090142d23f980b5 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:28.120 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d2ffff700 1 -- 172.21.15.117:0/440840611 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d34004160 con 0x7f5d3812f190 2023-01-25T06:49:28.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d2ffff700 1 -- 172.21.15.117:0/440840611 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d340042c0 con 0x7f5d3812f190 2023-01-25T06:49:28.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d2ffff700 1 -- 172.21.15.117:0/440840611 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d34004580 con 0x7f5d3812f190 2023-01-25T06:49:28.123 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 -- 172.21.15.117:0/440840611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3812f190 msgr2=0x7f5d38136d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.123 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 --2- 172.21.15.117:0/440840611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3812f190 0x7f5d38136d50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009890 tx=0x7f5d34009ba0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.123 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 -- 172.21.15.117:0/440840611 shutdown_connections 2023-01-25T06:49:28.124 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 --2- 172.21.15.117:0/440840611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3812f190 0x7f5d38136d50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.124 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.113+0000 7f5d3f751700 1 -- 172.21.15.117:0/440840611 >> 172.21.15.117:0/440840611 conn(0x7f5d380a5200 msgr2=0x7f5d380698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:28.124 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3f751700 1 -- 172.21.15.117:0/440840611 shutdown_connections 2023-01-25T06:49:28.124 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3f751700 1 -- 172.21.15.117:0/440840611 wait complete. 2023-01-25T06:49:28.124 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3f751700 1 Processor -- start 2023-01-25T06:49:28.125 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3f751700 1 -- start start 2023-01-25T06:49:28.125 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3f751700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d38096e80 0x7f5d3809d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:28.125 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3f751700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3809a070 con 0x7f5d38096e80 2023-01-25T06:49:28.125 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3d4ed700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d38096e80 0x7f5d3809d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:28.125 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3d4ed700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d38096e80 0x7f5d3809d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49048/0 (socket says 172.21.15.117:49048) 2023-01-25T06:49:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3d4ed700 1 -- 172.21.15.117:0/811058137 learned_addr learned my addr 172.21.15.117:0/811058137 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3d4ed700 1 -- 172.21.15.117:0/811058137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d34009570 con 0x7f5d38096e80 2023-01-25T06:49:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3d4ed700 1 --2- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d38096e80 0x7f5d3809d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800c480 tx=0x7f5d2800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d2dffb700 1 -- 172.21.15.117:0/811058137 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d28007410 con 0x7f5d38096e80 2023-01-25T06:49:28.126 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d2dffb700 1 -- 172.21.15.117:0/811058137 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28007570 con 0x7f5d38096e80 2023-01-25T06:49:28.127 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d2dffb700 1 -- 172.21.15.117:0/811058137 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d28005850 con 0x7f5d38096e80 2023-01-25T06:49:28.127 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d38097600 con 0x7f5d38096e80 2023-01-25T06:49:28.127 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.117+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3812e460 con 0x7f5d38096e80 2023-01-25T06:49:28.128 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.121+0000 7f5d2dffb700 1 -- 172.21.15.117:0/811058137 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f5d28019030 con 0x7f5d38096e80 2023-01-25T06:49:28.128 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.121+0000 7f5d2dffb700 1 --2- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5d1c033d80 0x7f5d1c036230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:28.128 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.121+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d38097d30 con 0x7f5d38096e80 2023-01-25T06:49:28.129 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.121+0000 7f5d2dffb700 1 -- 172.21.15.117:0/811058137 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2554+0+0 (secure 0 0 0) 0x7f5d28044530 con 0x7f5d38096e80 2023-01-25T06:49:28.129 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.121+0000 7f5d3ccec700 1 --2- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5d1c033d80 0x7f5d1c036230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:28.129 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.121+0000 7f5d3ccec700 1 --2- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5d1c033d80 0x7f5d1c036230 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009890 tx=0x7f5d34008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:28.132 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.129+0000 7f5d2dffb700 1 -- 172.21.15.117:0/811058137 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f5d280710a0 con 0x7f5d38096e80 2023-01-25T06:49:28.295 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d3809c390 con 0x7f5d38096e80 2023-01-25T06:49:28.296 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d2dffb700 1 -- 172.21.15.117:0/811058137 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v13) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d28017030 con 0x7f5d38096e80 2023-01-25T06:49:28.296 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 8 2023-01-25T06:49:28.298 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5d1c033d80 msgr2=0x7f5d1c036230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.299 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 --2- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5d1c033d80 0x7f5d1c036230 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009890 tx=0x7f5d34008030 comp rx=0 tx=0).stop 2023-01-25T06:49:28.299 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d38096e80 msgr2=0x7f5d3809d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.299 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 --2- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d38096e80 0x7f5d3809d420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800c480 tx=0x7f5d2800c840 comp rx=0 tx=0).stop 2023-01-25T06:49:28.299 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 shutdown_connections 2023-01-25T06:49:28.299 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 --2- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5d1c033d80 0x7f5d1c036230 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.300 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 --2- 172.21.15.117:0/811058137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d38096e80 0x7f5d3809d420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.300 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 >> 172.21.15.117:0/811058137 conn(0x7f5d380a5200 msgr2=0x7f5d3812c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:28.300 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 shutdown_connections 2023-01-25T06:49:28.300 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.293+0000 7f5d3f751700 1 -- 172.21.15.117:0/811058137 wait complete. 2023-01-25T06:49:28.310 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-01-25T06:49:28.420 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 Processor -- start 2023-01-25T06:49:28.421 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 -- start start 2023-01-25T06:49:28.421 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb6c0982f0 0x7fdb6c09ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:28.421 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6c097dc0 con 0x7fdb6c0698e0 2023-01-25T06:49:28.421 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6c09a420 con 0x7fdb6c0982f0 2023-01-25T06:49:28.423 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb70a0e700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fdb6c0698e0 0x7fdb6c069cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:42042/0 (socket says 172.21.15.117:42042) 2023-01-25T06:49:28.423 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb70a0e700 1 -- 172.21.15.117:0/2079571152 learned_addr learned my addr 172.21.15.117:0/2079571152 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:28.424 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb7120f700 1 --2- 172.21.15.117:0/2079571152 >> v2:172.21.15.117:3300/0 conn(0x7fdb6c0982f0 0x7fdb6c09ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:28.424 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb6bfff700 1 -- 172.21.15.117:0/2079571152 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953600514 0 0) 0x7fdb6c097dc0 con 0x7fdb6c0698e0 2023-01-25T06:49:28.424 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb6bfff700 1 -- 172.21.15.117:0/2079571152 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb540035d0 con 0x7fdb6c0698e0 2023-01-25T06:49:28.424 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb7120f700 1 -- 172.21.15.117:0/2079571152 >> v1:172.21.15.117:6789/0 conn(0x7fdb6c0698e0 legacy=0x7fdb6c069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.425 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb7120f700 1 -- 172.21.15.117:0/2079571152 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6c09a110 con 0x7fdb6c0982f0 2023-01-25T06:49:28.425 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb7120f700 1 --2- 172.21.15.117:0/2079571152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0982f0 0x7fdb6c09ce00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb600097c0 tx=0x7fdb60009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c509e919072886d4 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:28.425 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb6bfff700 1 -- 172.21.15.117:0/2079571152 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb60005d80 con 0x7fdb6c0982f0 2023-01-25T06:49:28.425 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb6bfff700 1 -- 172.21.15.117:0/2079571152 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb60004030 con 0x7fdb6c0982f0 2023-01-25T06:49:28.426 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb6bfff700 1 -- 172.21.15.117:0/2079571152 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb600042d0 con 0x7fdb6c0982f0 2023-01-25T06:49:28.426 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 -- 172.21.15.117:0/2079571152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0982f0 msgr2=0x7fdb6c09ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.426 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 --2- 172.21.15.117:0/2079571152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0982f0 0x7fdb6c09ce00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb600097c0 tx=0x7fdb60009ad0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.426 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 -- 172.21.15.117:0/2079571152 shutdown_connections 2023-01-25T06:49:28.427 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 --2- 172.21.15.117:0/2079571152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0982f0 0x7fdb6c09ce00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.427 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 -- 172.21.15.117:0/2079571152 >> 172.21.15.117:0/2079571152 conn(0x7fdb6c09e020 msgr2=0x7fdb6c09e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:28.427 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.417+0000 7fdb72211700 1 -- 172.21.15.117:0/2079571152 shutdown_connections 2023-01-25T06:49:28.427 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb72211700 1 -- 172.21.15.117:0/2079571152 wait complete. 2023-01-25T06:49:28.429 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb72211700 1 Processor -- start 2023-01-25T06:49:28.429 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb72211700 1 -- start start 2023-01-25T06:49:28.430 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb72211700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0698e0 0x7fdb6c070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:28.430 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb72211700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c06f070 con 0x7fdb6c0698e0 2023-01-25T06:49:28.430 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb7120f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0698e0 0x7fdb6c070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:28.430 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb7120f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0698e0 0x7fdb6c070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49080/0 (socket says 172.21.15.117:49080) 2023-01-25T06:49:28.430 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb7120f700 1 -- 172.21.15.117:0/1261749230 learned_addr learned my addr 172.21.15.117:0/1261749230 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:28.431 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb7120f700 1 -- 172.21.15.117:0/1261749230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb60009450 con 0x7fdb6c0698e0 2023-01-25T06:49:28.431 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb7120f700 1 --2- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0698e0 0x7fdb6c070420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c12dcf0 tx=0x7fdb60004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:28.431 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb69ffb700 1 -- 172.21.15.117:0/1261749230 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb60003e10 con 0x7fdb6c0698e0 2023-01-25T06:49:28.431 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb69ffb700 1 -- 172.21.15.117:0/1261749230 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6001c420 con 0x7fdb6c0698e0 2023-01-25T06:49:28.432 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6c098de0 con 0x7fdb6c0698e0 2023-01-25T06:49:28.432 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb69ffb700 1 -- 172.21.15.117:0/1261749230 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdb6001c6c0 con 0x7fdb6c0698e0 2023-01-25T06:49:28.432 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.421+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6c09d900 con 0x7fdb6c0698e0 2023-01-25T06:49:28.433 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.425+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6c09d340 con 0x7fdb6c0698e0 2023-01-25T06:49:28.433 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.425+0000 7fdb69ffb700 1 -- 172.21.15.117:0/1261749230 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7fdb6002c070 con 0x7fdb6c0698e0 2023-01-25T06:49:28.433 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.425+0000 7fdb69ffb700 1 --2- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fdb58033de0 0x7fdb58036290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:28.433 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.425+0000 7fdb69ffb700 1 -- 172.21.15.117:0/1261749230 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2554+0+0 (secure 0 0 0) 0x7fdb60046330 con 0x7fdb6c0698e0 2023-01-25T06:49:28.433 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.425+0000 7fdb70a0e700 1 --2- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fdb58033de0 0x7fdb58036290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:28.434 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.425+0000 7fdb70a0e700 1 --2- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fdb58033de0 0x7fdb58036290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb640012e0 tx=0x7fdb64001420 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:28.437 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.433+0000 7fdb69ffb700 1 -- 172.21.15.117:0/1261749230 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7fdb600243b0 con 0x7fdb6c0698e0 2023-01-25T06:49:28.600 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdb6c070e80 con 0x7fdb6c0698e0 2023-01-25T06:49:28.601 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb69ffb700 1 -- 172.21.15.117:0/1261749230 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v13) v1 ==== 84+0+64 (secure 0 0 0) 0x7fdb60016030 con 0x7fdb6c0698e0 2023-01-25T06:49:28.601 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:28.601 INFO:teuthology.orchestra.run.smithi117.stdout:[ 2023-01-25T06:49:28.602 INFO:teuthology.orchestra.run.smithi117.stdout: { 2023-01-25T06:49:28.602 INFO:teuthology.orchestra.run.smithi117.stdout: "poolnum": 1, 2023-01-25T06:49:28.602 INFO:teuthology.orchestra.run.smithi117.stdout: "poolname": "rbd" 2023-01-25T06:49:28.602 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-01-25T06:49:28.602 INFO:teuthology.orchestra.run.smithi117.stdout:] 2023-01-25T06:49:28.604 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fdb58033de0 msgr2=0x7fdb58036290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.604 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 --2- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fdb58033de0 0x7fdb58036290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb640012e0 tx=0x7fdb64001420 comp rx=0 tx=0).stop 2023-01-25T06:49:28.604 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0698e0 msgr2=0x7fdb6c070420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.605 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 --2- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0698e0 0x7fdb6c070420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c12dcf0 tx=0x7fdb60004770 comp rx=0 tx=0).stop 2023-01-25T06:49:28.605 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 shutdown_connections 2023-01-25T06:49:28.605 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 --2- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fdb58033de0 0x7fdb58036290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.606 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 --2- 172.21.15.117:0/1261749230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb6c0698e0 0x7fdb6c070420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.606 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 >> 172.21.15.117:0/1261749230 conn(0x7fdb6c09e020 msgr2=0x7fdb6c09b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:28.606 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.597+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 shutdown_connections 2023-01-25T06:49:28.607 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.601+0000 7fdb72211700 1 -- 172.21.15.117:0/1261749230 wait complete. 2023-01-25T06:49:28.615 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-01-25T06:49:28.616 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-01-25T06:49:28.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.729+0000 7f4ba09b8700 1 Processor -- start 2023-01-25T06:49:28.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.729+0000 7f4ba09b8700 1 -- start start 2023-01-25T06:49:28.739 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4b9c12bc00 0x7f4b9c12ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:28.739 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9c12c000 con 0x7f4b9c12b830 2023-01-25T06:49:28.739 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b9c12b670 con 0x7f4b9c12bc00 2023-01-25T06:49:28.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4b9c12bc00 0x7f4b9c12ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:28.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4b9c12bc00 0x7f4b9c12ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49088/0 (socket says 172.21.15.117:49088) 2023-01-25T06:49:28.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9b7fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4b9c12b830 0x7f4b9c12e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:42050/0 (socket says 172.21.15.117:42050) 2023-01-25T06:49:28.741 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9affd700 1 -- 172.21.15.117:0/2841942454 learned_addr learned my addr 172.21.15.117:0/2841942454 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:28.741 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9affd700 1 -- 172.21.15.117:0/2841942454 >> v1:172.21.15.117:6789/0 conn(0x7f4b9c12b830 legacy=0x7f4b9c12e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.741 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9affd700 1 -- 172.21.15.117:0/2841942454 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c099610 con 0x7f4b9c12bc00 2023-01-25T06:49:28.741 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9affd700 1 --2- 172.21.15.117:0/2841942454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12bc00 0x7f4b9c12ecd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009890 tx=0x7f4b90009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc804cbe248324e server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:28.741 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9a7fc700 1 -- 172.21.15.117:0/2841942454 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b90004160 con 0x7f4b9c12bc00 2023-01-25T06:49:28.742 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9a7fc700 1 -- 172.21.15.117:0/2841942454 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b900042c0 con 0x7f4b9c12bc00 2023-01-25T06:49:28.742 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4b9a7fc700 1 -- 172.21.15.117:0/2841942454 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b90004580 con 0x7f4b9c12bc00 2023-01-25T06:49:28.742 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 -- 172.21.15.117:0/2841942454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12bc00 msgr2=0x7f4b9c12ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:28.742 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 --2- 172.21.15.117:0/2841942454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12bc00 0x7f4b9c12ecd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009890 tx=0x7f4b90009ba0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 -- 172.21.15.117:0/2841942454 shutdown_connections 2023-01-25T06:49:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 --2- 172.21.15.117:0/2841942454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12bc00 0x7f4b9c12ecd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 -- 172.21.15.117:0/2841942454 >> 172.21.15.117:0/2841942454 conn(0x7f4b9c06ddc0 msgr2=0x7f4b9c0701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 -- 172.21.15.117:0/2841942454 shutdown_connections 2023-01-25T06:49:28.743 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 -- 172.21.15.117:0/2841942454 wait complete. 2023-01-25T06:49:28.744 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 Processor -- start 2023-01-25T06:49:28.744 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.733+0000 7f4ba09b8700 1 -- start start 2023-01-25T06:49:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4ba09b8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12b830 0x7f4b9c127f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4ba09b8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c09d200 con 0x7f4b9c12b830 2023-01-25T06:49:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b9b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12b830 0x7f4b9c127f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:28.746 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b9b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12b830 0x7f4b9c127f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49100/0 (socket says 172.21.15.117:49100) 2023-01-25T06:49:28.747 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b9b7fe700 1 -- 172.21.15.117:0/1944197229 learned_addr learned my addr 172.21.15.117:0/1944197229 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:28.747 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b9b7fe700 1 -- 172.21.15.117:0/1944197229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b90009570 con 0x7f4b9c12b830 2023-01-25T06:49:28.747 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b9b7fe700 1 --2- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12b830 0x7f4b9c127f10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00c480 tx=0x7f4b8c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:28.748 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b83fff700 1 -- 172.21.15.117:0/1944197229 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c005c00 con 0x7f4b9c12b830 2023-01-25T06:49:28.748 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b83fff700 1 -- 172.21.15.117:0/1944197229 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8c005d60 con 0x7f4b9c12b830 2023-01-25T06:49:28.748 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c099ff0 con 0x7f4b9c12b830 2023-01-25T06:49:28.748 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b83fff700 1 -- 172.21.15.117:0/1944197229 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c0070d0 con 0x7f4b9c12b830 2023-01-25T06:49:28.748 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c09ad90 con 0x7f4b9c12b830 2023-01-25T06:49:28.749 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b83fff700 1 -- 172.21.15.117:0/1944197229 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f4b8c019030 con 0x7f4b9c12b830 2023-01-25T06:49:28.749 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9c09a860 con 0x7f4b9c12b830 2023-01-25T06:49:28.749 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b83fff700 1 --2- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f4b88033d90 0x7f4b88036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:28.749 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b83fff700 1 -- 172.21.15.117:0/1944197229 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2554+0+0 (secure 0 0 0) 0x7f4b8c0444c0 con 0x7f4b9c12b830 2023-01-25T06:49:28.749 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.737+0000 7f4b9affd700 1 --2- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f4b88033d90 0x7f4b88036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.741+0000 7f4b9affd700 1 --2- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f4b88033d90 0x7f4b88036240 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009fd0 tx=0x7f4b90008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:28.750 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.745+0000 7f4b83fff700 1 -- 172.21.15.117:0/1944197229 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f4b8c019890 con 0x7f4b9c12b830 2023-01-25T06:49:28.920 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:28.917+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4b9c04ebb0 con 0x7f4b9c12b830 2023-01-25T06:49:29.714 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.709+0000 7f4b83fff700 1 -- 172.21.15.117:0/1944197229 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v14) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4b8c017030 con 0x7f4b9c12b830 2023-01-25T06:49:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.717+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f4b88033d90 msgr2=0x7f4b88036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:29.724 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.717+0000 7f4ba09b8700 1 --2- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f4b88033d90 0x7f4b88036240 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009fd0 tx=0x7f4b90008030 comp rx=0 tx=0).stop 2023-01-25T06:49:29.724 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.717+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12b830 msgr2=0x7f4b9c127f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:29.724 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.717+0000 7f4ba09b8700 1 --2- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12b830 0x7f4b9c127f10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00c480 tx=0x7f4b8c00c840 comp rx=0 tx=0).stop 2023-01-25T06:49:29.724 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.717+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 shutdown_connections 2023-01-25T06:49:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.717+0000 7f4ba09b8700 1 --2- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f4b88033d90 0x7f4b88036240 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.717+0000 7f4ba09b8700 1 --2- 172.21.15.117:0/1944197229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c12b830 0x7f4b9c127f10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.717+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 >> 172.21.15.117:0/1944197229 conn(0x7f4b9c06ddc0 msgr2=0x7f4b9c0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.721+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 shutdown_connections 2023-01-25T06:49:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.721+0000 7f4ba09b8700 1 -- 172.21.15.117:0/1944197229 wait complete. 2023-01-25T06:49:29.727 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_metadata' created 2023-01-25T06:49:29.739 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-01-25T06:49:29.858 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.853+0000 7f72f1c10700 1 Processor -- start 2023-01-25T06:49:29.859 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.853+0000 7f72f1c10700 1 -- start start 2023-01-25T06:49:29.860 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.853+0000 7f72f1c10700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f72ec0698e0 0x7f72ec069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:29.860 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.853+0000 7f72f1c10700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ec09d950 con 0x7f72ec06a1f0 2023-01-25T06:49:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.853+0000 7f72f1c10700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ec0978c0 con 0x7f72ec0698e0 2023-01-25T06:49:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f0c0e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f72ec0698e0 0x7f72ec069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72ebfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f72ec06a1f0 0x7f72ec136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:42062/0 (socket says 172.21.15.117:42062) 2023-01-25T06:49:29.861 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72ebfff700 1 -- 172.21.15.117:0/4047860006 learned_addr learned my addr 172.21.15.117:0/4047860006 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:29.862 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f0c0e700 1 -- 172.21.15.117:0/4047860006 >> v1:172.21.15.117:6789/0 conn(0x7f72ec06a1f0 legacy=0x7f72ec136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:29.862 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f0c0e700 1 -- 172.21.15.117:0/4047860006 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec0996d0 con 0x7f72ec0698e0 2023-01-25T06:49:29.862 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f0c0e700 1 --2- 172.21.15.117:0/4047860006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 0x7f72ec069cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f72e00097d0 tx=0x7f72e0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c978c40d0ca48a server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:29.862 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72eb7fe700 1 -- 172.21.15.117:0/4047860006 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72e00041c0 con 0x7f72ec0698e0 2023-01-25T06:49:29.864 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72eb7fe700 1 -- 172.21.15.117:0/4047860006 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e0004320 con 0x7f72ec0698e0 2023-01-25T06:49:29.864 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72eb7fe700 1 -- 172.21.15.117:0/4047860006 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72e00036c0 con 0x7f72ec0698e0 2023-01-25T06:49:29.867 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 -- 172.21.15.117:0/4047860006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 msgr2=0x7f72ec069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:29.867 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 --2- 172.21.15.117:0/4047860006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 0x7f72ec069cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f72e00097d0 tx=0x7f72e0009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:29.867 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 -- 172.21.15.117:0/4047860006 shutdown_connections 2023-01-25T06:49:29.867 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 --2- 172.21.15.117:0/4047860006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 0x7f72ec069cb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:29.867 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 -- 172.21.15.117:0/4047860006 >> 172.21.15.117:0/4047860006 conn(0x7f72ec09dfb0 msgr2=0x7f72ec09e380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:29.868 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 -- 172.21.15.117:0/4047860006 shutdown_connections 2023-01-25T06:49:29.868 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 -- 172.21.15.117:0/4047860006 wait complete. 2023-01-25T06:49:29.868 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 Processor -- start 2023-01-25T06:49:29.868 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 -- start start 2023-01-25T06:49:29.868 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 0x7f72ec1bbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:29.869 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f1c10700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1a8590 con 0x7f72ec0698e0 2023-01-25T06:49:29.869 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f0c0e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 0x7f72ec1bbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:29.869 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f0c0e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 0x7f72ec1bbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49116/0 (socket says 172.21.15.117:49116) 2023-01-25T06:49:29.869 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.857+0000 7f72f0c0e700 1 -- 172.21.15.117:0/4223574779 learned_addr learned my addr 172.21.15.117:0/4223574779 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:29.869 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72f0c0e700 1 -- 172.21.15.117:0/4223574779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e00094b0 con 0x7f72ec0698e0 2023-01-25T06:49:29.870 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72f0c0e700 1 --2- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 0x7f72ec1bbea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009b20 tx=0x7f72e0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:29.870 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72e97fa700 1 -- 172.21.15.117:0/4223574779 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72e0006ea0 con 0x7f72ec0698e0 2023-01-25T06:49:29.870 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72e97fa700 1 -- 172.21.15.117:0/4223574779 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e0011630 con 0x7f72ec0698e0 2023-01-25T06:49:29.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec097fe0 con 0x7f72ec0698e0 2023-01-25T06:49:29.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72e97fa700 1 -- 172.21.15.117:0/4223574779 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72e00118d0 con 0x7f72ec0698e0 2023-01-25T06:49:29.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec0993c0 con 0x7f72ec0698e0 2023-01-25T06:49:29.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72e97fa700 1 -- 172.21.15.117:0/4223574779 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f72e000f050 con 0x7f72ec0698e0 2023-01-25T06:49:29.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72e97fa700 1 --2- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f72d8033d40 0x7f72d80361f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:29.872 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72e97fa700 1 -- 172.21.15.117:0/4223574779 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2924+0+0 (secure 0 0 0) 0x7f72e0046570 con 0x7f72ec0698e0 2023-01-25T06:49:29.872 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72ebfff700 1 --2- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f72d8033d40 0x7f72d80361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:29.872 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec071ca0 con 0x7f72ec0698e0 2023-01-25T06:49:29.872 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.861+0000 7f72ebfff700 1 --2- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f72d8033d40 0x7f72d80361f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0016a0 tx=0x7f72dc007b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:29.874 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:29.869+0000 7f72e97fa700 1 -- 172.21.15.117:0/4223574779 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f72e0021020 con 0x7f72ec0698e0 2023-01-25T06:49:30.034 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.029+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f72ec04ebb0 con 0x7f72ec0698e0 2023-01-25T06:49:30.721 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.717+0000 7f72e97fa700 1 -- 172.21.15.117:0/4223574779 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v15) v1 ==== 170+0+0 (secure 0 0 0) 0x7f72e00340d0 con 0x7f72ec0698e0 2023-01-25T06:49:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.725+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f72d8033d40 msgr2=0x7f72d80361f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.729+0000 7f72f1c10700 1 --2- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f72d8033d40 0x7f72d80361f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0016a0 tx=0x7f72dc007b70 comp rx=0 tx=0).stop 2023-01-25T06:49:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.729+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 msgr2=0x7f72ec1bbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.729+0000 7f72f1c10700 1 --2- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 0x7f72ec1bbea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009b20 tx=0x7f72e0004770 comp rx=0 tx=0).stop 2023-01-25T06:49:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.729+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 shutdown_connections 2023-01-25T06:49:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.729+0000 7f72f1c10700 1 --2- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f72d8033d40 0x7f72d80361f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.729+0000 7f72f1c10700 1 --2- 172.21.15.117:0/4223574779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72ec0698e0 0x7f72ec1bbea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.729+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 >> 172.21.15.117:0/4223574779 conn(0x7f72ec09dfb0 msgr2=0x7f72ec1bd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.729+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 shutdown_connections 2023-01-25T06:49:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.729+0000 7f72f1c10700 1 -- 172.21.15.117:0/4223574779 wait complete. 2023-01-25T06:49:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data' created 2023-01-25T06:49:30.758 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-01-25T06:49:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 Processor -- start 2023-01-25T06:49:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 -- start start 2023-01-25T06:49:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f38b009d690 0x7f38b0134d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b0070ca0 con 0x7f38b009cbb0 2023-01-25T06:49:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b006ff40 con 0x7f38b009d690 2023-01-25T06:49:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38ae59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f38b009d690 0x7f38b0134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38ae59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f38b009d690 0x7f38b0134d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49118/0 (socket says 172.21.15.117:49118) 2023-01-25T06:49:30.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38ae59c700 1 -- 172.21.15.117:0/3978331508 learned_addr learned my addr 172.21.15.117:0/3978331508 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:30.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38ae59c700 1 -- 172.21.15.117:0/3978331508 >> v1:172.21.15.117:6789/0 conn(0x7f38b009cbb0 legacy=0x7f38b009cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:30.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38ae59c700 1 -- 172.21.15.117:0/3978331508 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b006fd20 con 0x7f38b009d690 2023-01-25T06:49:30.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38ae59c700 1 --2- 172.21.15.117:0/3978331508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009d690 0x7f38b0134d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009890 tx=0x7f38a0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9ec54e4edc943a server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:30.871 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38add9b700 1 -- 172.21.15.117:0/3978331508 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38a0004160 con 0x7f38b009d690 2023-01-25T06:49:30.872 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38add9b700 1 -- 172.21.15.117:0/3978331508 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a00042c0 con 0x7f38b009d690 2023-01-25T06:49:30.872 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38add9b700 1 -- 172.21.15.117:0/3978331508 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38a0004580 con 0x7f38b009d690 2023-01-25T06:49:30.875 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 -- 172.21.15.117:0/3978331508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009d690 msgr2=0x7f38b0134d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:30.876 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 --2- 172.21.15.117:0/3978331508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009d690 0x7f38b0134d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009890 tx=0x7f38a0009ba0 comp rx=0 tx=0).stop 2023-01-25T06:49:30.876 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 -- 172.21.15.117:0/3978331508 shutdown_connections 2023-01-25T06:49:30.876 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 --2- 172.21.15.117:0/3978331508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009d690 0x7f38b0134d50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:30.876 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 -- 172.21.15.117:0/3978331508 >> 172.21.15.117:0/3978331508 conn(0x7f38b009e020 msgr2=0x7f38b009e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:30.876 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 -- 172.21.15.117:0/3978331508 shutdown_connections 2023-01-25T06:49:30.877 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.865+0000 7f38b4804700 1 -- 172.21.15.117:0/3978331508 wait complete. 2023-01-25T06:49:30.877 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38b4804700 1 Processor -- start 2023-01-25T06:49:30.877 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38b4804700 1 -- start start 2023-01-25T06:49:30.877 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38b4804700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009cbb0 0x7f38b01206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:30.878 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38b4804700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0120dc0 con 0x7f38b009cbb0 2023-01-25T06:49:30.878 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38aed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009cbb0 0x7f38b01206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:30.878 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38aed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009cbb0 0x7f38b01206e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49122/0 (socket says 172.21.15.117:49122) 2023-01-25T06:49:30.878 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38aed9d700 1 -- 172.21.15.117:0/1360951330 learned_addr learned my addr 172.21.15.117:0/1360951330 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:30.878 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38aed9d700 1 -- 172.21.15.117:0/1360951330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0009570 con 0x7f38b009cbb0 2023-01-25T06:49:30.879 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38aed9d700 1 --2- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009cbb0 0x7f38b01206e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f389800c480 tx=0x7f389800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:30.879 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38a77fe700 1 -- 172.21.15.117:0/1360951330 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3898005410 con 0x7f38b009cbb0 2023-01-25T06:49:30.879 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38a77fe700 1 -- 172.21.15.117:0/1360951330 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898005570 con 0x7f38b009cbb0 2023-01-25T06:49:30.880 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b00a0e30 con 0x7f38b009cbb0 2023-01-25T06:49:30.880 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38a77fe700 1 -- 172.21.15.117:0/1360951330 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f38980037f0 con 0x7f38b009cbb0 2023-01-25T06:49:30.880 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b00705a0 con 0x7f38b009cbb0 2023-01-25T06:49:30.880 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.869+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b0070fe0 con 0x7f38b009cbb0 2023-01-25T06:49:30.881 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.873+0000 7f38a77fe700 1 -- 172.21.15.117:0/1360951330 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f3898019030 con 0x7f38b009cbb0 2023-01-25T06:49:30.882 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.873+0000 7f38a77fe700 1 --2- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3890033d80 0x7f3890036230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:30.882 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.873+0000 7f38a77fe700 1 -- 172.21.15.117:0/1360951330 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3306+0+0 (secure 0 0 0) 0x7f3898044910 con 0x7f38b009cbb0 2023-01-25T06:49:30.882 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.873+0000 7f38ae59c700 1 --2- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3890033d80 0x7f3890036230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:30.883 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.873+0000 7f38ae59c700 1 --2- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3890033d80 0x7f3890036230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f38a0000c00 tx=0x7f38a0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:30.883 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:30.881+0000 7f38a77fe700 1 -- 172.21.15.117:0/1360951330 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f38980194a0 con 0x7f38b009cbb0 2023-01-25T06:49:31.080 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.077+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f38b0120250 con 0x7f38b009cbb0 2023-01-25T06:49:31.086 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.081+0000 7f38a77fe700 1 -- 172.21.15.117:0/1360951330 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v3) v1 ==== 172+0+0 (secure 0 0 0) 0x7f38980192c0 con 0x7f38b009cbb0 2023-01-25T06:49:31.095 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.089+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3890033d80 msgr2=0x7f3890036230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.095 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.089+0000 7f38b4804700 1 --2- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3890033d80 0x7f3890036230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f38a0000c00 tx=0x7f38a0008030 comp rx=0 tx=0).stop 2023-01-25T06:49:31.095 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.089+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009cbb0 msgr2=0x7f38b01206e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.096 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.089+0000 7f38b4804700 1 --2- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009cbb0 0x7f38b01206e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f389800c480 tx=0x7f389800c840 comp rx=0 tx=0).stop 2023-01-25T06:49:31.099 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.097+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 shutdown_connections 2023-01-25T06:49:31.100 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.097+0000 7f38b4804700 1 --2- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3890033d80 0x7f3890036230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.100 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.097+0000 7f38b4804700 1 --2- 172.21.15.117:0/1360951330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b009cbb0 0x7f38b01206e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.101 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.097+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 >> 172.21.15.117:0/1360951330 conn(0x7f38b009e020 msgr2=0x7f38b0098f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:31.102 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.097+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 shutdown_connections 2023-01-25T06:49:31.102 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.097+0000 7f38b4804700 1 -- 172.21.15.117:0/1360951330 wait complete. 2023-01-25T06:49:31.108 INFO:teuthology.orchestra.run.smithi117.stderr:new fs with metadata pool 2 and data pool 3 2023-01-25T06:49:31.122 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T06:49:31.223 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 Processor -- start 2023-01-25T06:49:31.223 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 -- start start 2023-01-25T06:49:31.223 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54880698e0 0x7f5488069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:31.224 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5488099780 con 0x7f548806a1f0 2023-01-25T06:49:31.224 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5488099270 con 0x7f54880698e0 2023-01-25T06:49:31.224 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548d945700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54880698e0 0x7f5488069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548d144700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f548806a1f0 0x7f54881360f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:49294/0 (socket says 172.21.15.117:49294) 2023-01-25T06:49:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548d945700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54880698e0 0x7f5488069cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38492/0 (socket says 172.21.15.117:38492) 2023-01-25T06:49:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548d144700 1 -- 172.21.15.117:0/4066412084 learned_addr learned my addr 172.21.15.117:0/4066412084 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548d945700 1 -- 172.21.15.117:0/4066412084 >> v1:172.21.15.117:6789/0 conn(0x7f548806a1f0 legacy=0x7f54881360f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.225 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548d945700 1 -- 172.21.15.117:0/4066412084 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548809d530 con 0x7f54880698e0 2023-01-25T06:49:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548d945700 1 --2- 172.21.15.117:0/4066412084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 0x7f5488069cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5480009de0 tx=0x7f548000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1010c54b39cebacf server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548c943700 1 -- 172.21.15.117:0/4066412084 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f548000e070 con 0x7f54880698e0 2023-01-25T06:49:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548c943700 1 -- 172.21.15.117:0/4066412084 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480003b70 con 0x7f54880698e0 2023-01-25T06:49:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548c943700 1 -- 172.21.15.117:0/4066412084 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5480003e50 con 0x7f54880698e0 2023-01-25T06:49:31.226 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 -- 172.21.15.117:0/4066412084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 msgr2=0x7f5488069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 --2- 172.21.15.117:0/4066412084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 0x7f5488069cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5480009de0 tx=0x7f548000b330 comp rx=0 tx=0).stop 2023-01-25T06:49:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 -- 172.21.15.117:0/4066412084 shutdown_connections 2023-01-25T06:49:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 --2- 172.21.15.117:0/4066412084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 0x7f5488069cb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 -- 172.21.15.117:0/4066412084 >> 172.21.15.117:0/4066412084 conn(0x7f548809e020 msgr2=0x7f548809e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:31.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 -- 172.21.15.117:0/4066412084 shutdown_connections 2023-01-25T06:49:31.228 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.217+0000 7f548e947700 1 -- 172.21.15.117:0/4066412084 wait complete. 2023-01-25T06:49:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548e947700 1 Processor -- start 2023-01-25T06:49:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548e947700 1 -- start start 2023-01-25T06:49:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548e947700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 0x7f54881ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548e947700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54881ab430 con 0x7f54880698e0 2023-01-25T06:49:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548d945700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 0x7f54881ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548d945700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 0x7f54881ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38498/0 (socket says 172.21.15.117:38498) 2023-01-25T06:49:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548d945700 1 -- 172.21.15.117:0/3820233748 learned_addr learned my addr 172.21.15.117:0/3820233748 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548d945700 1 -- 172.21.15.117:0/3820233748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480009a70 con 0x7f54880698e0 2023-01-25T06:49:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548d945700 1 --2- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 0x7f54881ab9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f548000b370 tx=0x7f5480004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f547e7fc700 1 -- 172.21.15.117:0/3820233748 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54800046f0 con 0x7f54880698e0 2023-01-25T06:49:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f547e7fc700 1 -- 172.21.15.117:0/3820233748 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480004850 con 0x7f54880698e0 2023-01-25T06:49:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5488097fe0 con 0x7f54880698e0 2023-01-25T06:49:31.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f547e7fc700 1 -- 172.21.15.117:0/3820233748 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5480012430 con 0x7f54880698e0 2023-01-25T06:49:31.233 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.221+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548809d8e0 con 0x7f54880698e0 2023-01-25T06:49:31.233 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.225+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5488099470 con 0x7f54880698e0 2023-01-25T06:49:31.233 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.225+0000 7f547e7fc700 1 -- 172.21.15.117:0/3820233748 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f5480012590 con 0x7f54880698e0 2023-01-25T06:49:31.233 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.225+0000 7f547e7fc700 1 --2- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f546c033d90 0x7f546c036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:31.233 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.225+0000 7f547e7fc700 1 -- 172.21.15.117:0/3820233748 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3406+0+0 (secure 0 0 0) 0x7f5480047650 con 0x7f54880698e0 2023-01-25T06:49:31.234 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.225+0000 7f548d144700 1 --2- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f546c033d90 0x7f546c036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:31.234 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.225+0000 7f548d144700 1 --2- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f546c033d90 0x7f546c036240 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5478000f60 tx=0x7f5478007b30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:31.234 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.229+0000 7f547e7fc700 1 -- 172.21.15.117:0/3820233748 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f5480012820 con 0x7f54880698e0 2023-01-25T06:49:31.400 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.397+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5488099f40 con 0x7f54880698e0 2023-01-25T06:49:31.401 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.397+0000 7f547e7fc700 1 -- 172.21.15.117:0/3820233748 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+9519 (secure 0 0 0) 0x7f5480034540 con 0x7f54880698e0 2023-01-25T06:49:31.401 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:31.402 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":16,"fsid":"52037823-4da1-437a-aebe-937b909d2d65","created":"2023-01-25T06:49:16.306989+0000","modified":"2023-01-25T06:49:31.080915+0000","last_up_change":"2023-01-25T06:49:23.663021+0000","last_in_change":"2023-01-25T06:49:17.386973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-25T06:49:25.037352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-01-25T06:49:28.960628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-01-25T06:49:30.073668+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12f2c557-9775-4b67-a383-f8362a26e4fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":25717},{"type":"v1","addr":"172.21.15.117:6801","nonce":25717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":25717},{"type":"v1","addr":"172.21.15.117:6803","nonce":25717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":25717},{"type":"v1","addr":"172.21.15.117:6807","nonce":25717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":25717},{"type":"v1","addr":"172.21.15.117:6805","nonce":25717}]},"public_addr":"172.21.15.117:6801/25717","cluster_addr":"172.21.15.117:6803/25717","heartbeat_back_addr":"172.21.15.117:6807/25717","heartbeat_front_addr":"172.21.15.117:6805/25717","state":["exists","up"]},{"osd":1,"uuid":"dd866452-4e42-4fdd-8635-12cead61e275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":25718},{"type":"v1","addr":"172.21.15.117:6809","nonce":25718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":25718},{"type":"v1","addr":"172.21.15.117:6811","nonce":25718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":25718},{"type":"v1","addr":"172.21.15.117:6815","nonce":25718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":25718},{"type":"v1","addr":"172.21.15.117:6813","nonce":25718}]},"public_addr":"172.21.15.117:6809/25718","cluster_addr":"172.21.15.117:6811/25718","heartbeat_back_addr":"172.21.15.117:6815/25718","heartbeat_front_addr":"172.21.15.117:6813/25718","state":["exists","up"]},{"osd":2,"uuid":"ede1a6e7-9acf-4606-835e-e3518ce94df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":25720},{"type":"v1","addr":"172.21.15.117:6817","nonce":25720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":25720},{"type":"v1","addr":"172.21.15.117:6819","nonce":25720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":25720},{"type":"v1","addr":"172.21.15.117:6823","nonce":25720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":25720},{"type":"v1","addr":"172.21.15.117:6821","nonce":25720}]},"public_addr":"172.21.15.117:6817/25720","cluster_addr":"172.21.15.117:6819/25720","heartbeat_back_addr":"172.21.15.117:6823/25720","heartbeat_front_addr":"172.21.15.117:6821/25720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.868106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.616222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.917565+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:49:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f546c033d90 msgr2=0x7f546c036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.404 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 --2- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f546c033d90 0x7f546c036240 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5478000f60 tx=0x7f5478007b30 comp rx=0 tx=0).stop 2023-01-25T06:49:31.405 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 msgr2=0x7f54881ab9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.405 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 --2- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 0x7f54881ab9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f548000b370 tx=0x7f5480004af0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.405 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 shutdown_connections 2023-01-25T06:49:31.405 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 --2- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f546c033d90 0x7f546c036240 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 --2- 172.21.15.117:0/3820233748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54880698e0 0x7f54881ab9d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 >> 172.21.15.117:0/3820233748 conn(0x7f548809e020 msgr2=0x7f54881be040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 shutdown_connections 2023-01-25T06:49:31.406 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.401+0000 7f548e947700 1 -- 172.21.15.117:0/3820233748 wait complete. 2023-01-25T06:49:31.416 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T06:49:31.528 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 Processor -- start 2023-01-25T06:49:31.528 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 -- start start 2023-01-25T06:49:31.528 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f12fc097ed0 0x7f12fc134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:31.529 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc09b760 con 0x7f12fc0973f0 2023-01-25T06:49:31.529 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc09c8f0 con 0x7f12fc097ed0 2023-01-25T06:49:31.529 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12fad9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f12fc0973f0 0x7f12fc0977c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:49302/0 (socket says 172.21.15.117:49302) 2023-01-25T06:49:31.529 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12fad9d700 1 -- 172.21.15.117:0/606060554 learned_addr learned my addr 172.21.15.117:0/606060554 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:31.530 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12fa59c700 1 --2- 172.21.15.117:0/606060554 >> v2:172.21.15.117:3300/0 conn(0x7f12fc097ed0 0x7f12fc134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:31.531 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12f9d9b700 1 -- 172.21.15.117:0/606060554 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649930412 0 0) 0x7f12fc09b760 con 0x7f12fc0973f0 2023-01-25T06:49:31.532 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12f9d9b700 1 -- 172.21.15.117:0/606060554 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e40035d0 con 0x7f12fc0973f0 2023-01-25T06:49:31.532 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12fa59c700 1 -- 172.21.15.117:0/606060554 >> v1:172.21.15.117:6789/0 conn(0x7f12fc0973f0 legacy=0x7f12fc0977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.532 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12fa59c700 1 -- 172.21.15.117:0/606060554 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc0a1000 con 0x7f12fc097ed0 2023-01-25T06:49:31.532 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12fa59c700 1 --2- 172.21.15.117:0/606060554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc097ed0 0x7f12fc134ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12f00097d0 tx=0x7f12f0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4231ee68b0e2c0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:31.532 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12f9d9b700 1 -- 172.21.15.117:0/606060554 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12f0004160 con 0x7f12fc097ed0 2023-01-25T06:49:31.533 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12f9d9b700 1 -- 172.21.15.117:0/606060554 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f00042c0 con 0x7f12fc097ed0 2023-01-25T06:49:31.533 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f12f9d9b700 1 -- 172.21.15.117:0/606060554 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12f00045a0 con 0x7f12fc097ed0 2023-01-25T06:49:31.533 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 -- 172.21.15.117:0/606060554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc097ed0 msgr2=0x7f12fc134ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.533 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 --2- 172.21.15.117:0/606060554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc097ed0 0x7f12fc134ec0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12f00097d0 tx=0x7f12f0009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 -- 172.21.15.117:0/606060554 shutdown_connections 2023-01-25T06:49:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 --2- 172.21.15.117:0/606060554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc097ed0 0x7f12fc134ec0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 -- 172.21.15.117:0/606060554 >> 172.21.15.117:0/606060554 conn(0x7f12fc06d9b0 msgr2=0x7f12fc06fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 -- 172.21.15.117:0/606060554 shutdown_connections 2023-01-25T06:49:31.534 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 -- 172.21.15.117:0/606060554 wait complete. 2023-01-25T06:49:31.535 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.525+0000 7f1301117700 1 Processor -- start 2023-01-25T06:49:31.535 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f1301117700 1 -- start start 2023-01-25T06:49:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f1301117700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0973f0 0x7f12fc1258d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f1301117700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc12a2c0 con 0x7f12fc0973f0 2023-01-25T06:49:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f12fad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0973f0 0x7f12fc1258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:31.537 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f12fad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0973f0 0x7f12fc1258d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38504/0 (socket says 172.21.15.117:38504) 2023-01-25T06:49:31.538 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f12fad9d700 1 -- 172.21.15.117:0/2999699055 learned_addr learned my addr 172.21.15.117:0/2999699055 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:31.538 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f12fad9d700 1 -- 172.21.15.117:0/2999699055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f00094b0 con 0x7f12fc0973f0 2023-01-25T06:49:31.538 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f12fad9d700 1 --2- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0973f0 0x7f12fc1258d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f12f400cb40 tx=0x7f12f400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:31.538 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f12e37fe700 1 -- 172.21.15.117:0/2999699055 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12f400a760 con 0x7f12fc0973f0 2023-01-25T06:49:31.538 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f12e37fe700 1 -- 172.21.15.117:0/2999699055 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f400a8c0 con 0x7f12fc0973f0 2023-01-25T06:49:31.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f12e37fe700 1 -- 172.21.15.117:0/2999699055 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12f400ab60 con 0x7f12fc0973f0 2023-01-25T06:49:31.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc06c5f0 con 0x7f12fc0973f0 2023-01-25T06:49:31.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc09ace0 con 0x7f12fc0973f0 2023-01-25T06:49:31.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.529+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc09b9e0 con 0x7f12fc0973f0 2023-01-25T06:49:31.539 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.533+0000 7f12e37fe700 1 -- 172.21.15.117:0/2999699055 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f12f4019030 con 0x7f12fc0973f0 2023-01-25T06:49:31.540 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.533+0000 7f12e37fe700 1 --2- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f12e8033d90 0x7f12e8036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:31.540 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.533+0000 7f12e37fe700 1 -- 172.21.15.117:0/2999699055 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3406+0+0 (secure 0 0 0) 0x7f12f4044e40 con 0x7f12fc0973f0 2023-01-25T06:49:31.540 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.533+0000 7f12fa59c700 1 --2- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f12e8033d90 0x7f12e8036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:31.540 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.533+0000 7f12fa59c700 1 --2- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f12e8033d90 0x7f12e8036240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006ee0 tx=0x7f12f0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:31.543 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.541+0000 7f12e37fe700 1 -- 172.21.15.117:0/2999699055 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f12f401b3b0 con 0x7f12fc0973f0 2023-01-25T06:49:31.701 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.697+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12fc066340 con 0x7f12fc0973f0 2023-01-25T06:49:31.703 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.697+0000 7f12e37fe700 1 -- 172.21.15.117:0/2999699055 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+9519 (secure 0 0 0) 0x7f12f4017030 con 0x7f12fc0973f0 2023-01-25T06:49:31.703 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:31.703 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":16,"fsid":"52037823-4da1-437a-aebe-937b909d2d65","created":"2023-01-25T06:49:16.306989+0000","modified":"2023-01-25T06:49:31.080915+0000","last_up_change":"2023-01-25T06:49:23.663021+0000","last_in_change":"2023-01-25T06:49:17.386973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-25T06:49:25.037352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-01-25T06:49:28.960628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-01-25T06:49:30.073668+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12f2c557-9775-4b67-a383-f8362a26e4fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":25717},{"type":"v1","addr":"172.21.15.117:6801","nonce":25717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":25717},{"type":"v1","addr":"172.21.15.117:6803","nonce":25717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":25717},{"type":"v1","addr":"172.21.15.117:6807","nonce":25717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":25717},{"type":"v1","addr":"172.21.15.117:6805","nonce":25717}]},"public_addr":"172.21.15.117:6801/25717","cluster_addr":"172.21.15.117:6803/25717","heartbeat_back_addr":"172.21.15.117:6807/25717","heartbeat_front_addr":"172.21.15.117:6805/25717","state":["exists","up"]},{"osd":1,"uuid":"dd866452-4e42-4fdd-8635-12cead61e275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":25718},{"type":"v1","addr":"172.21.15.117:6809","nonce":25718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":25718},{"type":"v1","addr":"172.21.15.117:6811","nonce":25718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":25718},{"type":"v1","addr":"172.21.15.117:6815","nonce":25718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":25718},{"type":"v1","addr":"172.21.15.117:6813","nonce":25718}]},"public_addr":"172.21.15.117:6809/25718","cluster_addr":"172.21.15.117:6811/25718","heartbeat_back_addr":"172.21.15.117:6815/25718","heartbeat_front_addr":"172.21.15.117:6813/25718","state":["exists","up"]},{"osd":2,"uuid":"ede1a6e7-9acf-4606-835e-e3518ce94df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":25720},{"type":"v1","addr":"172.21.15.117:6817","nonce":25720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":25720},{"type":"v1","addr":"172.21.15.117:6819","nonce":25720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":25720},{"type":"v1","addr":"172.21.15.117:6823","nonce":25720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":25720},{"type":"v1","addr":"172.21.15.117:6821","nonce":25720}]},"public_addr":"172.21.15.117:6817/25720","cluster_addr":"172.21.15.117:6819/25720","heartbeat_back_addr":"172.21.15.117:6823/25720","heartbeat_front_addr":"172.21.15.117:6821/25720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.868106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.616222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.917565+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:49:31.706 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f12e8033d90 msgr2=0x7f12e8036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.706 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 --2- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f12e8033d90 0x7f12e8036240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006ee0 tx=0x7f12f0008030 comp rx=0 tx=0).stop 2023-01-25T06:49:31.706 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0973f0 msgr2=0x7f12fc1258d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.706 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 --2- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0973f0 0x7f12fc1258d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f12f400cb40 tx=0x7f12f400cc20 comp rx=0 tx=0).stop 2023-01-25T06:49:31.707 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 shutdown_connections 2023-01-25T06:49:31.707 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 --2- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f12e8033d90 0x7f12e8036240 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.707 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 --2- 172.21.15.117:0/2999699055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12fc0973f0 0x7f12fc1258d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.707 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 >> 172.21.15.117:0/2999699055 conn(0x7f12fc06d9b0 msgr2=0x7f12fc1b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:31.708 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 shutdown_connections 2023-01-25T06:49:31.708 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.701+0000 7f1301117700 1 -- 172.21.15.117:0/2999699055 wait complete. 2023-01-25T06:49:31.718 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-01-25T06:49:31.838 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7e417700 1 Processor -- start 2023-01-25T06:49:31.838 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7e417700 1 -- start start 2023-01-25T06:49:31.838 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7e417700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2b78099f80 0x7f2b7812e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:31.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7e417700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7809a350 con 0x7f2b7812ee50 2023-01-25T06:49:31.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7e417700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7809b890 con 0x7f2b78099f80 2023-01-25T06:49:31.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7d415700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2b78099f80 0x7f2b7812e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:31.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7cc14700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f2b7812ee50 0x7f2b781318b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:49312/0 (socket says 172.21.15.117:49312) 2023-01-25T06:49:31.839 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7cc14700 1 -- 172.21.15.117:0/1904170738 learned_addr learned my addr 172.21.15.117:0/1904170738 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:31.841 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b6ffff700 1 -- 172.21.15.117:0/1904170738 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411465018 0 0) 0x7f2b74001490 con 0x7f2b7812ee50 2023-01-25T06:49:31.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b6ffff700 1 -- 172.21.15.117:0/1904170738 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b640035d0 con 0x7f2b7812ee50 2023-01-25T06:49:31.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7d415700 1 -- 172.21.15.117:0/1904170738 >> v1:172.21.15.117:6789/0 conn(0x7f2b7812ee50 legacy=0x7f2b781318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7d415700 1 -- 172.21.15.117:0/1904170738 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7812f260 con 0x7f2b78099f80 2023-01-25T06:49:31.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b7d415700 1 --2- 172.21.15.117:0/1904170738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 0x7f2b7812e910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b680097d0 tx=0x7f2b68009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1504d4398c34734 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:31.842 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b6ffff700 1 -- 172.21.15.117:0/1904170738 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b68004160 con 0x7f2b78099f80 2023-01-25T06:49:31.843 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b6ffff700 1 -- 172.21.15.117:0/1904170738 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b680042c0 con 0x7f2b78099f80 2023-01-25T06:49:31.843 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.833+0000 7f2b6ffff700 1 -- 172.21.15.117:0/1904170738 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b680045a0 con 0x7f2b78099f80 2023-01-25T06:49:31.843 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 -- 172.21.15.117:0/1904170738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 msgr2=0x7f2b7812e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:31.843 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 --2- 172.21.15.117:0/1904170738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 0x7f2b7812e910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2b680097d0 tx=0x7f2b68009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.844 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 -- 172.21.15.117:0/1904170738 shutdown_connections 2023-01-25T06:49:31.844 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 --2- 172.21.15.117:0/1904170738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 0x7f2b7812e910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:31.844 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 -- 172.21.15.117:0/1904170738 >> 172.21.15.117:0/1904170738 conn(0x7f2b7806a260 msgr2=0x7f2b780a4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:31.844 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 -- 172.21.15.117:0/1904170738 shutdown_connections 2023-01-25T06:49:31.844 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 -- 172.21.15.117:0/1904170738 wait complete. 2023-01-25T06:49:31.845 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 Processor -- start 2023-01-25T06:49:31.845 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 -- start start 2023-01-25T06:49:31.845 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 0x7f2b7812b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:31.845 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7809fa70 con 0x7f2b78099f80 2023-01-25T06:49:31.846 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7d415700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 0x7f2b7812b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:31.846 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7d415700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 0x7f2b7812b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38518/0 (socket says 172.21.15.117:38518) 2023-01-25T06:49:31.846 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7d415700 1 -- 172.21.15.117:0/3150814361 learned_addr learned my addr 172.21.15.117:0/3150814361 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:31.846 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7d415700 1 -- 172.21.15.117:0/3150814361 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680094b0 con 0x7f2b78099f80 2023-01-25T06:49:31.847 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7d415700 1 --2- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 0x7f2b7812b0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2b680090b0 tx=0x7f2b68004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:31.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b6dffb700 1 -- 172.21.15.117:0/3150814361 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b68003c80 con 0x7f2b78099f80 2023-01-25T06:49:31.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7809bab0 con 0x7f2b78099f80 2023-01-25T06:49:31.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7812e600 con 0x7f2b78099f80 2023-01-25T06:49:31.848 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b6dffb700 1 -- 172.21.15.117:0/3150814361 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6801f070 con 0x7f2b78099f80 2023-01-25T06:49:31.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.837+0000 7f2b6dffb700 1 -- 172.21.15.117:0/3150814361 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b68011630 con 0x7f2b78099f80 2023-01-25T06:49:31.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.841+0000 7f2b6dffb700 1 -- 172.21.15.117:0/3150814361 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f2b6802a020 con 0x7f2b78099f80 2023-01-25T06:49:31.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.841+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b58005100 con 0x7f2b78099f80 2023-01-25T06:49:31.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.841+0000 7f2b6dffb700 1 --2- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f2b5c033d80 0x7f2b5c036230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:31.849 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.841+0000 7f2b6dffb700 1 -- 172.21.15.117:0/3150814361 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3406+0+0 (secure 0 0 0) 0x7f2b68046e70 con 0x7f2b78099f80 2023-01-25T06:49:31.850 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.841+0000 7f2b7cc14700 1 --2- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f2b5c033d80 0x7f2b5c036230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:31.850 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.841+0000 7f2b7cc14700 1 --2- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f2b5c033d80 0x7f2b5c036230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b740020a0 tx=0x7f2b740014d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:31.851 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:31.849+0000 7f2b6dffb700 1 -- 172.21.15.117:0/3150814361 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f2b68019e20 con 0x7f2b78099f80 2023-01-25T06:49:32.049 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.045+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2b58005930 con 0x7f2b78099f80 2023-01-25T06:49:32.087 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.081+0000 7f2b6dffb700 1 -- 172.21.15.117:0/3150814361 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v4) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2b680193c0 con 0x7f2b78099f80 2023-01-25T06:49:32.091 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.085+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f2b5c033d80 msgr2=0x7f2b5c036230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:32.091 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.085+0000 7f2b7e417700 1 --2- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f2b5c033d80 0x7f2b5c036230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b740020a0 tx=0x7f2b740014d0 comp rx=0 tx=0).stop 2023-01-25T06:49:32.092 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.085+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 msgr2=0x7f2b7812b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:32.092 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.085+0000 7f2b7e417700 1 --2- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 0x7f2b7812b0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2b680090b0 tx=0x7f2b68004770 comp rx=0 tx=0).stop 2023-01-25T06:49:32.092 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.085+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 shutdown_connections 2023-01-25T06:49:32.092 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.085+0000 7f2b7e417700 1 --2- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f2b5c033d80 0x7f2b5c036230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:32.092 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.085+0000 7f2b7e417700 1 --2- 172.21.15.117:0/3150814361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b78099f80 0x7f2b7812b0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:32.093 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.085+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 >> 172.21.15.117:0/3150814361 conn(0x7f2b7806a260 msgr2=0x7f2b780a2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:32.093 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.089+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 shutdown_connections 2023-01-25T06:49:32.093 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.089+0000 7f2b7e417700 1 -- 172.21.15.117:0/3150814361 wait complete. 2023-01-25T06:49:32.107 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-01-25T06:49:32.108 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-01-25T06:49:32.219 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.213+0000 7fe2e1a8d700 1 Processor -- start 2023-01-25T06:49:32.219 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.213+0000 7fe2e1a8d700 1 -- start start 2023-01-25T06:49:32.221 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe2dc099f80 0x7fe2dc12e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:32.221 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc09a350 con 0x7fe2dc12ee50 2023-01-25T06:49:32.222 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc09b890 con 0x7fe2dc099f80 2023-01-25T06:49:32.222 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e0a8b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe2dc099f80 0x7fe2dc12e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:32.222 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2dbfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe2dc12ee50 0x7fe2dc1318b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:49328/0 (socket says 172.21.15.117:49328) 2023-01-25T06:49:32.222 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2dbfff700 1 -- 172.21.15.117:0/1721315393 learned_addr learned my addr 172.21.15.117:0/1721315393 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:32.223 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2db7fe700 1 -- 172.21.15.117:0/1721315393 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211811597 0 0) 0x7fe2c4001110 con 0x7fe2dc12ee50 2023-01-25T06:49:32.223 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2db7fe700 1 -- 172.21.15.117:0/1721315393 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c80035d0 con 0x7fe2dc12ee50 2023-01-25T06:49:32.223 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e0a8b700 1 -- 172.21.15.117:0/1721315393 >> v1:172.21.15.117:6789/0 conn(0x7fe2dc12ee50 legacy=0x7fe2dc1318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:32.223 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e0a8b700 1 -- 172.21.15.117:0/1721315393 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc12f260 con 0x7fe2dc099f80 2023-01-25T06:49:32.224 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e0a8b700 1 --2- 172.21.15.117:0/1721315393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 0x7fe2dc12e910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0097d0 tx=0x7fe2cc009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b57a64da176717 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:32.224 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2db7fe700 1 -- 172.21.15.117:0/1721315393 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2cc004160 con 0x7fe2dc099f80 2023-01-25T06:49:32.225 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2db7fe700 1 -- 172.21.15.117:0/1721315393 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc0042c0 con 0x7fe2dc099f80 2023-01-25T06:49:32.225 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2db7fe700 1 -- 172.21.15.117:0/1721315393 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2cc004580 con 0x7fe2dc099f80 2023-01-25T06:49:32.226 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/1721315393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 msgr2=0x7fe2dc12e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:32.226 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 --2- 172.21.15.117:0/1721315393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 0x7fe2dc12e910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0097d0 tx=0x7fe2cc009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:32.226 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/1721315393 shutdown_connections 2023-01-25T06:49:32.226 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 --2- 172.21.15.117:0/1721315393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 0x7fe2dc12e910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:32.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/1721315393 >> 172.21.15.117:0/1721315393 conn(0x7fe2dc06a260 msgr2=0x7fe2dc0a4630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:32.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/1721315393 shutdown_connections 2023-01-25T06:49:32.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.217+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/1721315393 wait complete. 2023-01-25T06:49:32.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e1a8d700 1 Processor -- start 2023-01-25T06:49:32.227 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e1a8d700 1 -- start start 2023-01-25T06:49:32.228 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e1a8d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 0x7fe2dc12b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:32.228 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e1a8d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc09fa70 con 0x7fe2dc099f80 2023-01-25T06:49:32.228 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e0a8b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 0x7fe2dc12b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:32.228 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e0a8b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 0x7fe2dc12b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38534/0 (socket says 172.21.15.117:38534) 2023-01-25T06:49:32.228 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e0a8b700 1 -- 172.21.15.117:0/2848824632 learned_addr learned my addr 172.21.15.117:0/2848824632 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:32.229 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e0a8b700 1 -- 172.21.15.117:0/2848824632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc0094b0 con 0x7fe2dc099f80 2023-01-25T06:49:32.230 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e0a8b700 1 --2- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 0x7fe2dc12b0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0090b0 tx=0x7fe2cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:32.230 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2d97fa700 1 -- 172.21.15.117:0/2848824632 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2cc003c80 con 0x7fe2dc099f80 2023-01-25T06:49:32.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc09bab0 con 0x7fe2dc099f80 2023-01-25T06:49:32.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2d97fa700 1 -- 172.21.15.117:0/2848824632 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc01f070 con 0x7fe2dc099f80 2023-01-25T06:49:32.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2d97fa700 1 -- 172.21.15.117:0/2848824632 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2cc011630 con 0x7fe2dc099f80 2023-01-25T06:49:32.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.221+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc12e600 con 0x7fe2dc099f80 2023-01-25T06:49:32.231 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.225+0000 7fe2d97fa700 1 -- 172.21.15.117:0/2848824632 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7fe2cc02a020 con 0x7fe2dc099f80 2023-01-25T06:49:32.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.225+0000 7fe2d97fa700 1 --2- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe2bc033d30 0x7fe2bc0361e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:32.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.225+0000 7fe2d97fa700 1 -- 172.21.15.117:0/2848824632 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3406+0+0 (secure 0 0 0) 0x7fe2cc046c80 con 0x7fe2dc099f80 2023-01-25T06:49:32.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.225+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc09fca0 con 0x7fe2dc099f80 2023-01-25T06:49:32.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.225+0000 7fe2dbfff700 1 --2- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe2bc033d30 0x7fe2bc0361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:32.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.225+0000 7fe2dbfff700 1 --2- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe2bc033d30 0x7fe2bc0361e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40016c0 tx=0x7fe2c4001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:32.234 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.229+0000 7fe2d97fa700 1 -- 172.21.15.117:0/2848824632 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7fe2cc01a460 con 0x7fe2dc099f80 2023-01-25T06:49:32.430 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:32.425+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe2dc09a8c0 con 0x7fe2dc099f80 2023-01-25T06:49:33.286 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.281+0000 7fe2d97fa700 1 -- 172.21.15.117:0/2848824632 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v5) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe2cc010e60 con 0x7fe2dc099f80 2023-01-25T06:49:33.290 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe2bc033d30 msgr2=0x7fe2bc0361e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 --2- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe2bc033d30 0x7fe2bc0361e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40016c0 tx=0x7fe2c4001150 comp rx=0 tx=0).stop 2023-01-25T06:49:33.291 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 msgr2=0x7fe2dc12b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 --2- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 0x7fe2dc12b0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0090b0 tx=0x7fe2cc004770 comp rx=0 tx=0).stop 2023-01-25T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 shutdown_connections 2023-01-25T06:49:33.292 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 --2- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fe2bc033d30 0x7fe2bc0361e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 --2- 172.21.15.117:0/2848824632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2dc099f80 0x7fe2dc12b0d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 >> 172.21.15.117:0/2848824632 conn(0x7fe2dc06a260 msgr2=0x7fe2dc0a2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:33.293 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 shutdown_connections 2023-01-25T06:49:33.294 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.285+0000 7fe2e1a8d700 1 -- 172.21.15.117:0/2848824632 wait complete. 2023-01-25T06:49:33.306 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-01-25T06:49:33.306 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-01-25T06:49:33.423 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.417+0000 7ff0af59e700 1 Processor -- start 2023-01-25T06:49:33.424 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 -- start start 2023-01-25T06:49:33.426 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff0b006a3c0 0x7ff0b0134b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:33.426 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b006dc50 con 0x7ff0b00698e0 2023-01-25T06:49:33.426 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b006cef0 con 0x7ff0b006a3c0 2023-01-25T06:49:33.426 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0a5d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff0b006a3c0 0x7ff0b0134b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:33.426 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0a5d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff0b006a3c0 0x7ff0b0134b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38546/0 (socket says 172.21.15.117:38546) 2023-01-25T06:49:33.427 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0a5d9b700 1 -- 172.21.15.117:0/131225755 learned_addr learned my addr 172.21.15.117:0/131225755 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:33.427 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0ae59c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7ff0b00698e0 0x7ff0b0069cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:49332/0 (socket says 172.21.15.117:49332) 2023-01-25T06:49:33.427 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0a5d9b700 1 -- 172.21.15.117:0/131225755 >> v1:172.21.15.117:6789/0 conn(0x7ff0b00698e0 legacy=0x7ff0b0069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.427 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0add9b700 1 -- 172.21.15.117:0/131225755 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902833295 0 0) 0x7ff0b006dc50 con 0x7ff0b00698e0 2023-01-25T06:49:33.428 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0a5d9b700 1 -- 172.21.15.117:0/131225755 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b0071e60 con 0x7ff0b006a3c0 2023-01-25T06:49:33.428 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0a5d9b700 1 --2- 172.21.15.117:0/131225755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b006a3c0 0x7ff0b0134b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff098009890 tx=0x7ff098009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12ccc80039c561c server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:33.428 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0add9b700 1 -- 172.21.15.117:0/131225755 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff098004160 con 0x7ff0b006a3c0 2023-01-25T06:49:33.428 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0add9b700 1 -- 172.21.15.117:0/131225755 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0980042c0 con 0x7ff0b006a3c0 2023-01-25T06:49:33.428 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0add9b700 1 -- 172.21.15.117:0/131225755 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff098004580 con 0x7ff0b006a3c0 2023-01-25T06:49:33.430 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 -- 172.21.15.117:0/131225755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b006a3c0 msgr2=0x7ff0b0134b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.430 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 --2- 172.21.15.117:0/131225755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b006a3c0 0x7ff0b0134b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff098009890 tx=0x7ff098009ba0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.431 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 -- 172.21.15.117:0/131225755 shutdown_connections 2023-01-25T06:49:33.431 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 --2- 172.21.15.117:0/131225755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b006a3c0 0x7ff0b0134b40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.431 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 -- 172.21.15.117:0/131225755 >> 172.21.15.117:0/131225755 conn(0x7ff0b009e020 msgr2=0x7ff0b009e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:33.432 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 -- 172.21.15.117:0/131225755 shutdown_connections 2023-01-25T06:49:33.432 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.421+0000 7ff0af59e700 1 -- 172.21.15.117:0/131225755 wait complete. 2023-01-25T06:49:33.433 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0af59e700 1 Processor -- start 2023-01-25T06:49:33.433 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0af59e700 1 -- start start 2023-01-25T06:49:33.433 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0af59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b00698e0 0x7ff0b0121780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:33.434 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0af59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b011e4b0 con 0x7ff0b00698e0 2023-01-25T06:49:33.434 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0ae59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b00698e0 0x7ff0b0121780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:33.434 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0ae59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b00698e0 0x7ff0b0121780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38556/0 (socket says 172.21.15.117:38556) 2023-01-25T06:49:33.435 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0ae59c700 1 -- 172.21.15.117:0/234150627 learned_addr learned my addr 172.21.15.117:0/234150627 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:33.435 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0ae59c700 1 -- 172.21.15.117:0/234150627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098009570 con 0x7ff0b00698e0 2023-01-25T06:49:33.435 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0ae59c700 1 --2- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b00698e0 0x7ff0b0121780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c420 tx=0x7ff0a000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:33.436 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0a77fe700 1 -- 172.21.15.117:0/234150627 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0a0005b90 con 0x7ff0b00698e0 2023-01-25T06:49:33.436 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0a77fe700 1 -- 172.21.15.117:0/234150627 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a0005cf0 con 0x7ff0b00698e0 2023-01-25T06:49:33.436 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b006d860 con 0x7ff0b00698e0 2023-01-25T06:49:33.437 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0a77fe700 1 -- 172.21.15.117:0/234150627 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0a00070d0 con 0x7ff0b00698e0 2023-01-25T06:49:33.437 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.425+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b006ccd0 con 0x7ff0b00698e0 2023-01-25T06:49:33.439 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.429+0000 7ff0a77fe700 1 -- 172.21.15.117:0/234150627 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7ff0a0019030 con 0x7ff0b00698e0 2023-01-25T06:49:33.439 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.429+0000 7ff0a77fe700 1 --2- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7ff094033d90 0x7ff094036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:33.440 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.429+0000 7ff0a77fe700 1 -- 172.21.15.117:0/234150627 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3406+0+0 (secure 0 0 0) 0x7ff0a00448b0 con 0x7ff0b00698e0 2023-01-25T06:49:33.440 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.429+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b006d550 con 0x7ff0b00698e0 2023-01-25T06:49:33.440 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.429+0000 7ff0a5d9b700 1 --2- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7ff094033d90 0x7ff094036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:33.441 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.429+0000 7ff0a5d9b700 1 --2- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7ff094033d90 0x7ff094036240 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff098000c00 tx=0x7ff098008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:33.442 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.437+0000 7ff0a77fe700 1 -- 172.21.15.117:0/234150627 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7ff0a0017020 con 0x7ff0b00698e0 2023-01-25T06:49:33.634 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.629+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0b0130880 con 0x7ff0b00698e0 2023-01-25T06:49:33.635 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.629+0000 7ff0a77fe700 1 -- 172.21.15.117:0/234150627 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2348 (secure 0 0 0) 0x7ff0a00196d0 con 0x7ff0b00698e0 2023-01-25T06:49:33.635 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:33.635 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-01-25T06:49:31.081224+0000","modified":"2023-01-25T06:49:33.280770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4145":{"gid":4145,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6827/2542771210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2542771210},{"type":"v1","addr":"172.21.15.117:6827","nonce":2542771210}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-01-25T06:49:33.637 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7ff094033d90 msgr2=0x7ff094036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.637 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 --2- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7ff094033d90 0x7ff094036240 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff098000c00 tx=0x7ff098008030 comp rx=0 tx=0).stop 2023-01-25T06:49:33.638 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b00698e0 msgr2=0x7ff0b0121780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.638 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 --2- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b00698e0 0x7ff0b0121780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000c420 tx=0x7ff0a000c7e0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.638 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 shutdown_connections 2023-01-25T06:49:33.638 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 --2- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7ff094033d90 0x7ff094036240 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.638 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 --2- 172.21.15.117:0/234150627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff0b00698e0 0x7ff0b0121780 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.639 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 >> 172.21.15.117:0/234150627 conn(0x7ff0b009e020 msgr2=0x7ff0b006f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:33.639 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 shutdown_connections 2023-01-25T06:49:33.639 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.633+0000 7ff0af59e700 1 -- 172.21.15.117:0/234150627 wait complete. 2023-01-25T06:49:33.640 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2023-01-25T06:49:33.649 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T06:49:33.761 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0e438700 1 Processor -- start 2023-01-25T06:49:33.761 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0e438700 1 -- start start 2023-01-25T06:49:33.762 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0e438700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3d0812e880 0x7f3d0812ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:33.763 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0e438700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0812e680 con 0x7f3d080982b0 2023-01-25T06:49:33.763 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0e438700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0809edb0 con 0x7f3d0812e880 2023-01-25T06:49:33.763 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0d436700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3d0812e880 0x7f3d0812ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:33.764 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0d436700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3d0812e880 0x7f3d0812ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38570/0 (socket says 172.21.15.117:38570) 2023-01-25T06:49:33.764 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0d436700 1 -- 172.21.15.117:0/4207656977 learned_addr learned my addr 172.21.15.117:0/4207656977 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:33.764 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0cc35700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3d080982b0 0x7f3d080986f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:49338/0 (socket says 172.21.15.117:49338) 2023-01-25T06:49:33.764 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0d436700 1 -- 172.21.15.117:0/4207656977 >> v1:172.21.15.117:6789/0 conn(0x7f3d080982b0 legacy=0x7f3d080986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0d436700 1 -- 172.21.15.117:0/4207656977 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d080994d0 con 0x7f3d0812e880 2023-01-25T06:49:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d0d436700 1 --2- 172.21.15.117:0/4207656977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d0812e880 0x7f3d0812ec50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009e50 tx=0x7f3d0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5626bd62ba1dc696 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d07fff700 1 -- 172.21.15.117:0/4207656977 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d0000e070 con 0x7f3d0812e880 2023-01-25T06:49:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d07fff700 1 -- 172.21.15.117:0/4207656977 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d00003a40 con 0x7f3d0812e880 2023-01-25T06:49:33.765 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.757+0000 7f3d07fff700 1 -- 172.21.15.117:0/4207656977 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d00003d00 con 0x7f3d0812e880 2023-01-25T06:49:33.768 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 -- 172.21.15.117:0/4207656977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d0812e880 msgr2=0x7f3d0812ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.768 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 --2- 172.21.15.117:0/4207656977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d0812e880 0x7f3d0812ec50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009e50 tx=0x7f3d0000b330 comp rx=0 tx=0).stop 2023-01-25T06:49:33.768 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 -- 172.21.15.117:0/4207656977 shutdown_connections 2023-01-25T06:49:33.769 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 --2- 172.21.15.117:0/4207656977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d0812e880 0x7f3d0812ec50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.769 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 -- 172.21.15.117:0/4207656977 >> 172.21.15.117:0/4207656977 conn(0x7f3d0806de50 msgr2=0x7f3d08070260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:33.769 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 -- 172.21.15.117:0/4207656977 shutdown_connections 2023-01-25T06:49:33.769 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 -- 172.21.15.117:0/4207656977 wait complete. 2023-01-25T06:49:33.770 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 Processor -- start 2023-01-25T06:49:33.770 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 -- start start 2023-01-25T06:49:33.770 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d080982b0 0x7f3d080a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:33.771 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08118dc0 con 0x7f3d080982b0 2023-01-25T06:49:33.771 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0d436700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d080982b0 0x7f3d080a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:33.772 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0d436700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d080982b0 0x7f3d080a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38580/0 (socket says 172.21.15.117:38580) 2023-01-25T06:49:33.772 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0d436700 1 -- 172.21.15.117:0/1080242824 learned_addr learned my addr 172.21.15.117:0/1080242824 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:33.772 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0d436700 1 -- 172.21.15.117:0/1080242824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d00009b30 con 0x7f3d080982b0 2023-01-25T06:49:33.773 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0d436700 1 --2- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d080982b0 0x7f3d080a04b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d00015040 tx=0x7f3d00004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:33.773 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d05ffb700 1 -- 172.21.15.117:0/1080242824 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d00004370 con 0x7f3d080982b0 2023-01-25T06:49:33.773 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d05ffb700 1 -- 172.21.15.117:0/1080242824 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d000044d0 con 0x7f3d080982b0 2023-01-25T06:49:33.774 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0812f190 con 0x7f3d080982b0 2023-01-25T06:49:33.774 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d05ffb700 1 -- 172.21.15.117:0/1080242824 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d000047b0 con 0x7f3d080982b0 2023-01-25T06:49:33.774 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.761+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0809ef90 con 0x7f3d080982b0 2023-01-25T06:49:33.775 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.765+0000 7f3d05ffb700 1 -- 172.21.15.117:0/1080242824 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f3d00022030 con 0x7f3d080982b0 2023-01-25T06:49:33.776 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.765+0000 7f3d05ffb700 1 --2- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3ce8033d90 0x7f3ce8036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:33.776 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.765+0000 7f3d05ffb700 1 -- 172.21.15.117:0/1080242824 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3406+0+0 (secure 0 0 0) 0x7f3d00046ec0 con 0x7f3d080982b0 2023-01-25T06:49:33.777 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.765+0000 7f3d0cc35700 1 --2- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3ce8033d90 0x7f3ce8036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:33.777 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.765+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d08099ce0 con 0x7f3d080982b0 2023-01-25T06:49:33.777 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.765+0000 7f3d0cc35700 1 --2- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3ce8033d90 0x7f3ce8036240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8001690 tx=0x7f3cf8007af0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:33.778 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.773+0000 7f3d05ffb700 1 -- 172.21.15.117:0/1080242824 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f3d0002a020 con 0x7f3d080982b0 2023-01-25T06:49:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.933+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d0812f4a0 con 0x7f3d080982b0 2023-01-25T06:49:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.933+0000 7f3d05ffb700 1 -- 172.21.15.117:0/1080242824 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+9506 (secure 0 0 0) 0x7f3d00017070 con 0x7f3d080982b0 2023-01-25T06:49:33.938 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:33.938 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":17,"fsid":"52037823-4da1-437a-aebe-937b909d2d65","created":"2023-01-25T06:49:16.306989+0000","modified":"2023-01-25T06:49:32.083262+0000","last_up_change":"2023-01-25T06:49:23.663021+0000","last_in_change":"2023-01-25T06:49:17.386973+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-25T06:49:25.037352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-01-25T06:49:28.960628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-01-25T06:49:30.073668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"12f2c557-9775-4b67-a383-f8362a26e4fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":25717},{"type":"v1","addr":"172.21.15.117:6801","nonce":25717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":25717},{"type":"v1","addr":"172.21.15.117:6803","nonce":25717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":25717},{"type":"v1","addr":"172.21.15.117:6807","nonce":25717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":25717},{"type":"v1","addr":"172.21.15.117:6805","nonce":25717}]},"public_addr":"172.21.15.117:6801/25717","cluster_addr":"172.21.15.117:6803/25717","heartbeat_back_addr":"172.21.15.117:6807/25717","heartbeat_front_addr":"172.21.15.117:6805/25717","state":["exists","up"]},{"osd":1,"uuid":"dd866452-4e42-4fdd-8635-12cead61e275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":25718},{"type":"v1","addr":"172.21.15.117:6809","nonce":25718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":25718},{"type":"v1","addr":"172.21.15.117:6811","nonce":25718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":25718},{"type":"v1","addr":"172.21.15.117:6815","nonce":25718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":25718},{"type":"v1","addr":"172.21.15.117:6813","nonce":25718}]},"public_addr":"172.21.15.117:6809/25718","cluster_addr":"172.21.15.117:6811/25718","heartbeat_back_addr":"172.21.15.117:6815/25718","heartbeat_front_addr":"172.21.15.117:6813/25718","state":["exists","up"]},{"osd":2,"uuid":"ede1a6e7-9acf-4606-835e-e3518ce94df6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":25720},{"type":"v1","addr":"172.21.15.117:6817","nonce":25720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":25720},{"type":"v1","addr":"172.21.15.117:6819","nonce":25720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":25720},{"type":"v1","addr":"172.21.15.117:6823","nonce":25720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":25720},{"type":"v1","addr":"172.21.15.117:6821","nonce":25720}]},"public_addr":"172.21.15.117:6817/25720","cluster_addr":"172.21.15.117:6819/25720","heartbeat_back_addr":"172.21.15.117:6823/25720","heartbeat_front_addr":"172.21.15.117:6821/25720","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.868106+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.616222+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:49:21.917565+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:49:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.933+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3ce8033d90 msgr2=0x7f3ce8036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.933+0000 7f3d0e438700 1 --2- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3ce8033d90 0x7f3ce8036240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8001690 tx=0x7f3cf8007af0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.933+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d080982b0 msgr2=0x7f3d080a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:33.940 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.933+0000 7f3d0e438700 1 --2- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d080982b0 0x7f3d080a04b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d00015040 tx=0x7f3d00004af0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.942 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.937+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 shutdown_connections 2023-01-25T06:49:33.942 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.937+0000 7f3d0e438700 1 --2- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f3ce8033d90 0x7f3ce8036240 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.937+0000 7f3d0e438700 1 --2- 172.21.15.117:0/1080242824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d080982b0 0x7f3d080a04b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.937+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 >> 172.21.15.117:0/1080242824 conn(0x7f3d0806de50 msgr2=0x7f3d0812ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.937+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 shutdown_connections 2023-01-25T06:49:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:33.937+0000 7f3d0e438700 1 -- 172.21.15.117:0/1080242824 wait complete. 2023-01-25T06:49:33.952 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-01-25T06:49:33.952 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T06:49:34.060 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 Processor -- start 2023-01-25T06:49:34.061 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 -- start start 2023-01-25T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f547009a000 0x7f5470131f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547009aa20 con 0x7f5470132450 2023-01-25T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5470132840 con 0x7f547009a000 2023-01-25T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f546f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f547009a000 0x7f5470131f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:34.062 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f546effd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f5470132450 0x7f547012e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:49344/0 (socket says 172.21.15.117:49344) 2023-01-25T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f546effd700 1 -- 172.21.15.117:0/3563078297 learned_addr learned my addr 172.21.15.117:0/3563078297 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f546f7fe700 1 -- 172.21.15.117:0/3563078297 >> v1:172.21.15.117:6789/0 conn(0x7f5470132450 legacy=0x7f547012e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:34.063 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f546f7fe700 1 -- 172.21.15.117:0/3563078297 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470131c00 con 0x7f547009a000 2023-01-25T06:49:34.064 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f546f7fe700 1 --2- 172.21.15.117:0/3563078297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 0x7f5470131f10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54600097d0 tx=0x7f5460009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41168ec15b95e0b server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f546e7fc700 1 -- 172.21.15.117:0/3563078297 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54600041c0 con 0x7f547009a000 2023-01-25T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f546e7fc700 1 -- 172.21.15.117:0/3563078297 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460004320 con 0x7f547009a000 2023-01-25T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f546e7fc700 1 -- 172.21.15.117:0/3563078297 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54600036c0 con 0x7f547009a000 2023-01-25T06:49:34.065 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 -- 172.21.15.117:0/3563078297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 msgr2=0x7f5470131f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 --2- 172.21.15.117:0/3563078297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 0x7f5470131f10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54600097d0 tx=0x7f5460009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 -- 172.21.15.117:0/3563078297 shutdown_connections 2023-01-25T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 --2- 172.21.15.117:0/3563078297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 0x7f5470131f10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.057+0000 7f54749e2700 1 -- 172.21.15.117:0/3563078297 >> 172.21.15.117:0/3563078297 conn(0x7f54700698e0 msgr2=0x7f5470069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:34.066 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f54749e2700 1 -- 172.21.15.117:0/3563078297 shutdown_connections 2023-01-25T06:49:34.067 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f54749e2700 1 -- 172.21.15.117:0/3563078297 wait complete. 2023-01-25T06:49:34.067 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f54749e2700 1 Processor -- start 2023-01-25T06:49:34.067 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f54749e2700 1 -- start start 2023-01-25T06:49:34.069 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f54749e2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 0x7f547012b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:34.069 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f54749e2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547009d300 con 0x7f547009a000 2023-01-25T06:49:34.069 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f546f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 0x7f547012b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:34.069 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f546f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 0x7f547012b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38590/0 (socket says 172.21.15.117:38590) 2023-01-25T06:49:34.070 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f546f7fe700 1 -- 172.21.15.117:0/2677001431 learned_addr learned my addr 172.21.15.117:0/2677001431 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:34.070 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f546f7fe700 1 -- 172.21.15.117:0/2677001431 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600094b0 con 0x7f547009a000 2023-01-25T06:49:34.070 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f546f7fe700 1 --2- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 0x7f547012b060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54600090b0 tx=0x7f5460004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:34.070 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f5467fff700 1 -- 172.21.15.117:0/2677001431 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5460011630 con 0x7f547009a000 2023-01-25T06:49:34.070 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f5467fff700 1 -- 172.21.15.117:0/2677001431 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460011790 con 0x7f547009a000 2023-01-25T06:49:34.071 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5470131850 con 0x7f547009a000 2023-01-25T06:49:34.071 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f5467fff700 1 -- 172.21.15.117:0/2677001431 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5460011a30 con 0x7f547009a000 2023-01-25T06:49:34.071 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.061+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547009bb30 con 0x7f547009a000 2023-01-25T06:49:34.071 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.065+0000 7f5467fff700 1 -- 172.21.15.117:0/2677001431 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7f546002d070 con 0x7f547009a000 2023-01-25T06:49:34.071 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.065+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547009aeb0 con 0x7f547009a000 2023-01-25T06:49:34.072 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.065+0000 7f5467fff700 1 --2- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5450033d80 0x7f5450036230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:34.072 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.065+0000 7f5467fff700 1 -- 172.21.15.117:0/2677001431 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3406+0+0 (secure 0 0 0) 0x7f5460046690 con 0x7f547009a000 2023-01-25T06:49:34.072 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.065+0000 7f546effd700 1 --2- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5450033d80 0x7f5450036230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:34.073 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.065+0000 7f546effd700 1 --2- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5450033d80 0x7f5450036230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5458000f60 tx=0x7f5458007b30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:34.077 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.073+0000 7f5467fff700 1 -- 172.21.15.117:0/2677001431 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7f5460024950 con 0x7f547009a000 2023-01-25T06:49:34.235 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.229+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 --> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54700a1d40 con 0x7f5450033d80 2023-01-25T06:49:34.250 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.245+0000 7f5467fff700 1 -- 172.21.15.117:0/2677001431 <== mgr.4101 v2:172.21.15.117:6824/25572 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+255635 (secure 0 0 0) 0x7f54700a1d40 con 0x7f5450033d80 2023-01-25T06:49:34.253 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:34.257 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.249+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5450033d80 msgr2=0x7f5450036230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:34.257 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.249+0000 7f54749e2700 1 --2- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5450033d80 0x7f5450036230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5458000f60 tx=0x7f5458007b30 comp rx=0 tx=0).stop 2023-01-25T06:49:34.258 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.249+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 msgr2=0x7f547012b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:34.258 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.249+0000 7f54749e2700 1 --2- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 0x7f547012b060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54600090b0 tx=0x7f5460004770 comp rx=0 tx=0).stop 2023-01-25T06:49:34.258 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.253+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 shutdown_connections 2023-01-25T06:49:34.258 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.253+0000 7f54749e2700 1 --2- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7f5450033d80 0x7f5450036230 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:34.258 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.253+0000 7f54749e2700 1 --2- 172.21.15.117:0/2677001431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f547009a000 0x7f547012b060 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:34.259 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.253+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 >> 172.21.15.117:0/2677001431 conn(0x7f54700698e0 msgr2=0x7f54701b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:34.259 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.253+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 shutdown_connections 2023-01-25T06:49:34.259 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.253+0000 7f54749e2700 1 -- 172.21.15.117:0/2677001431 wait complete. 2023-01-25T06:49:34.259 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2023-01-25T06:49:34.268 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-01-25T06:49:33.695106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":15572,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":15168,"kb_avail":314557228,"statfs":{"total":322122547200,"available":322106601472,"internally_reserved":0,"allocated":389120,"data_stored":98636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15532032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097401+0000","last_change":"2023-01-25T06:49:31.116814+0000","last_active":"2023-01-25T06:49:32.097401+0000","last_peered":"2023-01-25T06:49:32.097401+0000","last_clean":"2023-01-25T06:49:32.097401+0000","last_became_active":"2023-01-25T06:49:31.116493+0000","last_became_peered":"2023-01-25T06:49:31.116493+0000","last_unstale":"2023-01-25T06:49:32.097401+0000","last_undegraded":"2023-01-25T06:49:32.097401+0000","last_fullsized":"2023-01-25T06:49:32.097401+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:25:40.674339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097628+0000","last_change":"2023-01-25T06:49:31.118198+0000","last_active":"2023-01-25T06:49:32.097628+0000","last_peered":"2023-01-25T06:49:32.097628+0000","last_clean":"2023-01-25T06:49:32.097628+0000","last_became_active":"2023-01-25T06:49:31.117967+0000","last_became_peered":"2023-01-25T06:49:31.117967+0000","last_unstale":"2023-01-25T06:49:32.097628+0000","last_undegraded":"2023-01-25T06:49:32.097628+0000","last_fullsized":"2023-01-25T06:49:32.097628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:36:34.519717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096003+0000","last_change":"2023-01-25T06:49:31.110220+0000","last_active":"2023-01-25T06:49:32.096003+0000","last_peered":"2023-01-25T06:49:32.096003+0000","last_clean":"2023-01-25T06:49:32.096003+0000","last_became_active":"2023-01-25T06:49:31.109897+0000","last_became_peered":"2023-01-25T06:49:31.109897+0000","last_unstale":"2023-01-25T06:49:32.096003+0000","last_undegraded":"2023-01-25T06:49:32.096003+0000","last_fullsized":"2023-01-25T06:49:32.096003+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T08:03:02.699337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097873+0000","last_change":"2023-01-25T06:49:31.117725+0000","last_active":"2023-01-25T06:49:32.097873+0000","last_peered":"2023-01-25T06:49:32.097873+0000","last_clean":"2023-01-25T06:49:32.097873+0000","last_became_active":"2023-01-25T06:49:31.117358+0000","last_became_peered":"2023-01-25T06:49:31.117358+0000","last_unstale":"2023-01-25T06:49:32.097873+0000","last_undegraded":"2023-01-25T06:49:32.097873+0000","last_fullsized":"2023-01-25T06:49:32.097873+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:59:52.778738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096535+0000","last_change":"2023-01-25T06:49:31.110964+0000","last_active":"2023-01-25T06:49:32.096535+0000","last_peered":"2023-01-25T06:49:32.096535+0000","last_clean":"2023-01-25T06:49:32.096535+0000","last_became_active":"2023-01-25T06:49:31.110055+0000","last_became_peered":"2023-01-25T06:49:31.110055+0000","last_unstale":"2023-01-25T06:49:32.096535+0000","last_undegraded":"2023-01-25T06:49:32.096535+0000","last_fullsized":"2023-01-25T06:49:32.096535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T07:13:32.029208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097089+0000","last_change":"2023-01-25T06:49:31.118370+0000","last_active":"2023-01-25T06:49:32.097089+0000","last_peered":"2023-01-25T06:49:32.097089+0000","last_clean":"2023-01-25T06:49:32.097089+0000","last_became_active":"2023-01-25T06:49:31.117918+0000","last_became_peered":"2023-01-25T06:49:31.117918+0000","last_unstale":"2023-01-25T06:49:32.097089+0000","last_undegraded":"2023-01-25T06:49:32.097089+0000","last_fullsized":"2023-01-25T06:49:32.097089+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T11:23:37.727304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096453+0000","last_change":"2023-01-25T06:49:31.110594+0000","last_active":"2023-01-25T06:49:32.096453+0000","last_peered":"2023-01-25T06:49:32.096453+0000","last_clean":"2023-01-25T06:49:32.096453+0000","last_became_active":"2023-01-25T06:49:31.110364+0000","last_became_peered":"2023-01-25T06:49:31.110364+0000","last_unstale":"2023-01-25T06:49:32.096453+0000","last_undegraded":"2023-01-25T06:49:32.096453+0000","last_fullsized":"2023-01-25T06:49:32.096453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:25:28.364287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.095669+0000","last_change":"2023-01-25T06:49:31.114813+0000","last_active":"2023-01-25T06:49:32.095669+0000","last_peered":"2023-01-25T06:49:32.095669+0000","last_clean":"2023-01-25T06:49:32.095669+0000","last_became_active":"2023-01-25T06:49:31.109639+0000","last_became_peered":"2023-01-25T06:49:31.109639+0000","last_unstale":"2023-01-25T06:49:32.095669+0000","last_undegraded":"2023-01-25T06:49:32.095669+0000","last_fullsized":"2023-01-25T06:49:32.095669+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T17:00:39.780533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"16'1","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097977+0000","last_change":"2023-01-25T06:49:30.740907+0000","last_active":"2023-01-25T06:49:32.097977+0000","last_peered":"2023-01-25T06:49:32.097977+0000","last_clean":"2023-01-25T06:49:32.097977+0000","last_became_active":"2023-01-25T06:49:30.740649+0000","last_became_peered":"2023-01-25T06:49:30.740649+0000","last_unstale":"2023-01-25T06:49:32.097977+0000","last_undegraded":"2023-01-25T06:49:32.097977+0000","last_fullsized":"2023-01-25T06:49:32.097977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:26:21.705061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.098149+0000","last_change":"2023-01-25T06:49:31.118231+0000","last_active":"2023-01-25T06:49:32.098149+0000","last_peered":"2023-01-25T06:49:32.098149+0000","last_clean":"2023-01-25T06:49:32.098149+0000","last_became_active":"2023-01-25T06:49:31.117653+0000","last_became_peered":"2023-01-25T06:49:31.117653+0000","last_unstale":"2023-01-25T06:49:32.098149+0000","last_undegraded":"2023-01-25T06:49:32.098149+0000","last_fullsized":"2023-01-25T06:49:32.098149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T13:05:33.194760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097129+0000","last_change":"2023-01-25T06:49:30.750338+0000","last_active":"2023-01-25T06:49:32.097129+0000","last_peered":"2023-01-25T06:49:32.097129+0000","last_clean":"2023-01-25T06:49:32.097129+0000","last_became_active":"2023-01-25T06:49:30.749568+0000","last_became_peered":"2023-01-25T06:49:30.749568+0000","last_unstale":"2023-01-25T06:49:32.097129+0000","last_undegraded":"2023-01-25T06:49:32.097129+0000","last_fullsized":"2023-01-25T06:49:32.097129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:26:16.618730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096506+0000","last_change":"2023-01-25T06:49:31.110118+0000","last_active":"2023-01-25T06:49:32.096506+0000","last_peered":"2023-01-25T06:49:32.096506+0000","last_clean":"2023-01-25T06:49:32.096506+0000","last_became_active":"2023-01-25T06:49:31.109870+0000","last_became_peered":"2023-01-25T06:49:31.109870+0000","last_unstale":"2023-01-25T06:49:32.096506+0000","last_undegraded":"2023-01-25T06:49:32.096506+0000","last_fullsized":"2023-01-25T06:49:32.096506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T17:10:01.724076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096092+0000","last_change":"2023-01-25T06:49:30.751421+0000","last_active":"2023-01-25T06:49:32.096092+0000","last_peered":"2023-01-25T06:49:32.096092+0000","last_clean":"2023-01-25T06:49:32.096092+0000","last_became_active":"2023-01-25T06:49:30.750849+0000","last_became_peered":"2023-01-25T06:49:30.750849+0000","last_unstale":"2023-01-25T06:49:32.096092+0000","last_undegraded":"2023-01-25T06:49:32.096092+0000","last_fullsized":"2023-01-25T06:49:32.096092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T08:09:24.872588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T06:49:31.106724+0000","last_change":"2023-01-25T06:49:26.713319+0000","last_active":"2023-01-25T06:49:31.106724+0000","last_peered":"2023-01-25T06:49:31.106724+0000","last_clean":"2023-01-25T06:49:31.106724+0000","last_became_active":"2023-01-25T06:49:26.712953+0000","last_became_peered":"2023-01-25T06:49:26.712953+0000","last_unstale":"2023-01-25T06:49:31.106724+0000","last_undegraded":"2023-01-25T06:49:31.106724+0000","last_fullsized":"2023-01-25T06:49:31.106724+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_clean_scrub_stamp":"2023-01-25T06:49:25.677822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:41:43.668886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096686+0000","last_change":"2023-01-25T06:49:30.738522+0000","last_active":"2023-01-25T06:49:32.096686+0000","last_peered":"2023-01-25T06:49:32.096686+0000","last_clean":"2023-01-25T06:49:32.096686+0000","last_became_active":"2023-01-25T06:49:30.737145+0000","last_became_peered":"2023-01-25T06:49:30.737145+0000","last_unstale":"2023-01-25T06:49:32.096686+0000","last_undegraded":"2023-01-25T06:49:32.096686+0000","last_fullsized":"2023-01-25T06:49:32.096686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T18:14:58.917108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097296+0000","last_change":"2023-01-25T06:49:31.116020+0000","last_active":"2023-01-25T06:49:32.097296+0000","last_peered":"2023-01-25T06:49:32.097296+0000","last_clean":"2023-01-25T06:49:32.097296+0000","last_became_active":"2023-01-25T06:49:31.111050+0000","last_became_peered":"2023-01-25T06:49:31.111050+0000","last_unstale":"2023-01-25T06:49:32.097296+0000","last_undegraded":"2023-01-25T06:49:32.097296+0000","last_fullsized":"2023-01-25T06:49:32.097296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T12:22:25.444522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-01-25T06:49:26.713570+0000","last_change":"2023-01-25T06:49:26.708904+0000","last_active":"2023-01-25T06:49:26.713570+0000","last_peered":"2023-01-25T06:49:26.713570+0000","last_clean":"2023-01-25T06:49:26.713570+0000","last_became_active":"2023-01-25T06:49:26.708599+0000","last_became_peered":"2023-01-25T06:49:26.708599+0000","last_unstale":"2023-01-25T06:49:26.713570+0000","last_undegraded":"2023-01-25T06:49:26.713570+0000","last_fullsized":"2023-01-25T06:49:26.713570+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_clean_scrub_stamp":"2023-01-25T06:49:25.677822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T16:57:07.664557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-01-25T06:49:26.715634+0000","last_change":"2023-01-25T06:49:26.709216+0000","last_active":"2023-01-25T06:49:26.715634+0000","last_peered":"2023-01-25T06:49:26.715634+0000","last_clean":"2023-01-25T06:49:26.715634+0000","last_became_active":"2023-01-25T06:49:26.708841+0000","last_became_peered":"2023-01-25T06:49:26.708841+0000","last_unstale":"2023-01-25T06:49:26.715634+0000","last_undegraded":"2023-01-25T06:49:26.715634+0000","last_fullsized":"2023-01-25T06:49:26.715634+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_clean_scrub_stamp":"2023-01-25T06:49:25.677822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T13:24:08.355130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.095798+0000","last_change":"2023-01-25T06:49:31.115362+0000","last_active":"2023-01-25T06:49:32.095798+0000","last_peered":"2023-01-25T06:49:32.095798+0000","last_clean":"2023-01-25T06:49:32.095798+0000","last_became_active":"2023-01-25T06:49:31.115145+0000","last_became_peered":"2023-01-25T06:49:31.115145+0000","last_unstale":"2023-01-25T06:49:32.095798+0000","last_undegraded":"2023-01-25T06:49:32.095798+0000","last_fullsized":"2023-01-25T06:49:32.095798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T07:39:52.089438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T06:49:31.103464+0000","last_change":"2023-01-25T06:49:26.709528+0000","last_active":"2023-01-25T06:49:31.103464+0000","last_peered":"2023-01-25T06:49:31.103464+0000","last_clean":"2023-01-25T06:49:31.103464+0000","last_became_active":"2023-01-25T06:49:26.709135+0000","last_became_peered":"2023-01-25T06:49:26.709135+0000","last_unstale":"2023-01-25T06:49:31.103464+0000","last_undegraded":"2023-01-25T06:49:31.103464+0000","last_fullsized":"2023-01-25T06:49:31.103464+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_clean_scrub_stamp":"2023-01-25T06:49:25.677822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T11:37:04.722659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"16'2","reported_seq":13,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097971+0000","last_change":"2023-01-25T06:49:30.751468+0000","last_active":"2023-01-25T06:49:32.097971+0000","last_peered":"2023-01-25T06:49:32.097971+0000","last_clean":"2023-01-25T06:49:32.097971+0000","last_became_active":"2023-01-25T06:49:30.751118+0000","last_became_peered":"2023-01-25T06:49:30.751118+0000","last_unstale":"2023-01-25T06:49:32.097971+0000","last_undegraded":"2023-01-25T06:49:32.097971+0000","last_fullsized":"2023-01-25T06:49:32.097971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T16:00:19.066985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T06:49:31.107136+0000","last_change":"2023-01-25T06:49:26.713086+0000","last_active":"2023-01-25T06:49:31.107136+0000","last_peered":"2023-01-25T06:49:31.107136+0000","last_clean":"2023-01-25T06:49:31.107136+0000","last_became_active":"2023-01-25T06:49:26.712777+0000","last_became_peered":"2023-01-25T06:49:26.712777+0000","last_unstale":"2023-01-25T06:49:31.107136+0000","last_undegraded":"2023-01-25T06:49:31.107136+0000","last_fullsized":"2023-01-25T06:49:31.107136+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_clean_scrub_stamp":"2023-01-25T06:49:25.677822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:28:27.368620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097464+0000","last_change":"2023-01-25T06:49:30.751210+0000","last_active":"2023-01-25T06:49:32.097464+0000","last_peered":"2023-01-25T06:49:32.097464+0000","last_clean":"2023-01-25T06:49:32.097464+0000","last_became_active":"2023-01-25T06:49:30.750937+0000","last_became_peered":"2023-01-25T06:49:30.750937+0000","last_unstale":"2023-01-25T06:49:32.097464+0000","last_undegraded":"2023-01-25T06:49:32.097464+0000","last_fullsized":"2023-01-25T06:49:32.097464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T12:49:41.594396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-01-25T06:49:26.715948+0000","last_change":"2023-01-25T06:49:26.711463+0000","last_active":"2023-01-25T06:49:26.715948+0000","last_peered":"2023-01-25T06:49:26.715948+0000","last_clean":"2023-01-25T06:49:26.715948+0000","last_became_active":"2023-01-25T06:49:26.711279+0000","last_became_peered":"2023-01-25T06:49:26.711279+0000","last_unstale":"2023-01-25T06:49:26.715948+0000","last_undegraded":"2023-01-25T06:49:26.715948+0000","last_fullsized":"2023-01-25T06:49:26.715948+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_clean_scrub_stamp":"2023-01-25T06:49:25.677822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T08:23:28.126077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.098084+0000","last_change":"2023-01-25T06:49:30.750905+0000","last_active":"2023-01-25T06:49:32.098084+0000","last_peered":"2023-01-25T06:49:32.098084+0000","last_clean":"2023-01-25T06:49:32.098084+0000","last_became_active":"2023-01-25T06:49:30.749677+0000","last_became_peered":"2023-01-25T06:49:30.749677+0000","last_unstale":"2023-01-25T06:49:32.098084+0000","last_undegraded":"2023-01-25T06:49:32.098084+0000","last_fullsized":"2023-01-25T06:49:32.098084+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:41:51.723053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T06:49:31.103486+0000","last_change":"2023-01-25T06:49:26.713084+0000","last_active":"2023-01-25T06:49:31.103486+0000","last_peered":"2023-01-25T06:49:31.103486+0000","last_clean":"2023-01-25T06:49:31.103486+0000","last_became_active":"2023-01-25T06:49:26.712788+0000","last_became_peered":"2023-01-25T06:49:26.712788+0000","last_unstale":"2023-01-25T06:49:31.103486+0000","last_undegraded":"2023-01-25T06:49:31.103486+0000","last_fullsized":"2023-01-25T06:49:31.103486+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_clean_scrub_stamp":"2023-01-25T06:49:25.677822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T18:36:16.364119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097782+0000","last_change":"2023-01-25T06:49:31.118478+0000","last_active":"2023-01-25T06:49:32.097782+0000","last_peered":"2023-01-25T06:49:32.097782+0000","last_clean":"2023-01-25T06:49:32.097782+0000","last_became_active":"2023-01-25T06:49:31.117816+0000","last_became_peered":"2023-01-25T06:49:31.117816+0000","last_unstale":"2023-01-25T06:49:32.097782+0000","last_undegraded":"2023-01-25T06:49:32.097782+0000","last_fullsized":"2023-01-25T06:49:32.097782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:45:33.442522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-25T06:49:31.105562+0000","last_change":"2023-01-25T06:49:26.711411+0000","last_active":"2023-01-25T06:49:31.105562+0000","last_peered":"2023-01-25T06:49:31.105562+0000","last_clean":"2023-01-25T06:49:31.105562+0000","last_became_active":"2023-01-25T06:49:26.710978+0000","last_became_peered":"2023-01-25T06:49:26.710978+0000","last_unstale":"2023-01-25T06:49:31.105562+0000","last_undegraded":"2023-01-25T06:49:31.105562+0000","last_fullsized":"2023-01-25T06:49:31.105562+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:25.677822+0000","last_clean_scrub_stamp":"2023-01-25T06:49:25.677822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T17:29:42.818367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097241+0000","last_change":"2023-01-25T06:49:30.751650+0000","last_active":"2023-01-25T06:49:32.097241+0000","last_peered":"2023-01-25T06:49:32.097241+0000","last_clean":"2023-01-25T06:49:32.097241+0000","last_became_active":"2023-01-25T06:49:30.751349+0000","last_became_peered":"2023-01-25T06:49:30.751349+0000","last_unstale":"2023-01-25T06:49:32.097241+0000","last_undegraded":"2023-01-25T06:49:32.097241+0000","last_fullsized":"2023-01-25T06:49:32.097241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T16:34:09.736338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096823+0000","last_change":"2023-01-25T06:49:31.118615+0000","last_active":"2023-01-25T06:49:32.096823+0000","last_peered":"2023-01-25T06:49:32.096823+0000","last_clean":"2023-01-25T06:49:32.096823+0000","last_became_active":"2023-01-25T06:49:31.118062+0000","last_became_peered":"2023-01-25T06:49:31.118062+0000","last_unstale":"2023-01-25T06:49:32.096823+0000","last_undegraded":"2023-01-25T06:49:32.096823+0000","last_fullsized":"2023-01-25T06:49:32.096823+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T07:05:21.303445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"16'3","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097645+0000","last_change":"2023-01-25T06:49:30.751109+0000","last_active":"2023-01-25T06:49:32.097645+0000","last_peered":"2023-01-25T06:49:32.097645+0000","last_clean":"2023-01-25T06:49:32.097645+0000","last_became_active":"2023-01-25T06:49:30.749941+0000","last_became_peered":"2023-01-25T06:49:30.749941+0000","last_unstale":"2023-01-25T06:49:32.097645+0000","last_undegraded":"2023-01-25T06:49:32.097645+0000","last_fullsized":"2023-01-25T06:49:32.097645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T11:39:20.757763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097804+0000","last_change":"2023-01-25T06:49:30.740953+0000","last_active":"2023-01-25T06:49:32.097804+0000","last_peered":"2023-01-25T06:49:32.097804+0000","last_clean":"2023-01-25T06:49:32.097804+0000","last_became_active":"2023-01-25T06:49:30.740667+0000","last_became_peered":"2023-01-25T06:49:30.740667+0000","last_unstale":"2023-01-25T06:49:32.097804+0000","last_undegraded":"2023-01-25T06:49:32.097804+0000","last_fullsized":"2023-01-25T06:49:32.097804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:20:53.344368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.095592+0000","last_change":"2023-01-25T06:49:31.110686+0000","last_active":"2023-01-25T06:49:32.095592+0000","last_peered":"2023-01-25T06:49:32.095592+0000","last_clean":"2023-01-25T06:49:32.095592+0000","last_became_active":"2023-01-25T06:49:31.110455+0000","last_became_peered":"2023-01-25T06:49:31.110455+0000","last_unstale":"2023-01-25T06:49:32.095592+0000","last_undegraded":"2023-01-25T06:49:32.095592+0000","last_fullsized":"2023-01-25T06:49:32.095592+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T07:59:12.360151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096688+0000","last_change":"2023-01-25T06:49:31.114947+0000","last_active":"2023-01-25T06:49:32.096688+0000","last_peered":"2023-01-25T06:49:32.096688+0000","last_clean":"2023-01-25T06:49:32.096688+0000","last_became_active":"2023-01-25T06:49:31.111192+0000","last_became_peered":"2023-01-25T06:49:31.111192+0000","last_unstale":"2023-01-25T06:49:32.096688+0000","last_undegraded":"2023-01-25T06:49:32.096688+0000","last_fullsized":"2023-01-25T06:49:32.096688+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:08:17.760541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"16'1","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096689+0000","last_change":"2023-01-25T06:49:30.751212+0000","last_active":"2023-01-25T06:49:32.096689+0000","last_peered":"2023-01-25T06:49:32.096689+0000","last_clean":"2023-01-25T06:49:32.096689+0000","last_became_active":"2023-01-25T06:49:30.750951+0000","last_became_peered":"2023-01-25T06:49:30.750951+0000","last_unstale":"2023-01-25T06:49:32.096689+0000","last_undegraded":"2023-01-25T06:49:32.096689+0000","last_fullsized":"2023-01-25T06:49:32.096689+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:51:52.621904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097483+0000","last_change":"2023-01-25T06:49:31.118927+0000","last_active":"2023-01-25T06:49:32.097483+0000","last_peered":"2023-01-25T06:49:32.097483+0000","last_clean":"2023-01-25T06:49:32.097483+0000","last_became_active":"2023-01-25T06:49:31.117504+0000","last_became_peered":"2023-01-25T06:49:31.117504+0000","last_unstale":"2023-01-25T06:49:32.097483+0000","last_undegraded":"2023-01-25T06:49:32.097483+0000","last_fullsized":"2023-01-25T06:49:32.097483+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T13:57:43.609245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":34,"acting":34,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":22,"acting":22,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705666,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":106496,"data_stored":25791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":106496,"data_stored":25791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.2619999999999996}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5492,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":104852108,"statfs":{"total":107374182400,"available":107368558592,"internally_reserved":0,"allocated":176128,"data_stored":47054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T06:49:34.274 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb49274e700 1 Processor -- start 2023-01-25T06:49:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb49274e700 1 -- start start 2023-01-25T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb49274e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb48c12bc40 0x7fb48c133870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:34.386 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb49274e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c0a23b0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb49274e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c09b980 con 0x7fb48c12bc40 2023-01-25T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb49174c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb48c09b5b0 0x7fb48c12b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:49358/0 (socket says 172.21.15.117:49358) 2023-01-25T06:49:34.387 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb49174c700 1 -- 172.21.15.117:0/2980363283 learned_addr learned my addr 172.21.15.117:0/2980363283 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-01-25T06:49:34.388 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb490f4b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb48c12bc40 0x7fb48c133870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb483fff700 1 -- 172.21.15.117:0/2980363283 <== mon.0 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067604487 0 0) 0x7fb48c0a23b0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb483fff700 1 -- 172.21.15.117:0/2980363283 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4780035d0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb490f4b700 1 -- 172.21.15.117:0/2980363283 >> v1:172.21.15.117:6789/0 conn(0x7fb48c09b5b0 legacy=0x7fb48c12b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:34.390 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.381+0000 7fb490f4b700 1 -- 172.21.15.117:0/2980363283 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c09bc70 con 0x7fb48c12bc40 2023-01-25T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb490f4b700 1 --2- 172.21.15.117:0/2980363283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c12bc40 0x7fb48c133870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0097d0 tx=0x7fb47c009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e256ebd3e7f16fb server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb483fff700 1 -- 172.21.15.117:0/2980363283 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb47c004160 con 0x7fb48c12bc40 2023-01-25T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb483fff700 1 -- 172.21.15.117:0/2980363283 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c0042c0 con 0x7fb48c12bc40 2023-01-25T06:49:34.391 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb483fff700 1 -- 172.21.15.117:0/2980363283 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb47c004580 con 0x7fb48c12bc40 2023-01-25T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 -- 172.21.15.117:0/2980363283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c12bc40 msgr2=0x7fb48c133870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 --2- 172.21.15.117:0/2980363283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c12bc40 0x7fb48c133870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0097d0 tx=0x7fb47c009ae0 comp rx=0 tx=0).stop 2023-01-25T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 -- 172.21.15.117:0/2980363283 shutdown_connections 2023-01-25T06:49:34.392 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 --2- 172.21.15.117:0/2980363283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c12bc40 0x7fb48c133870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 -- 172.21.15.117:0/2980363283 >> 172.21.15.117:0/2980363283 conn(0x7fb48c06de50 msgr2=0x7fb48c070260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 -- 172.21.15.117:0/2980363283 shutdown_connections 2023-01-25T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 -- 172.21.15.117:0/2980363283 wait complete. 2023-01-25T06:49:34.393 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 Processor -- start 2023-01-25T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 -- start start 2023-01-25T06:49:34.395 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c09b5b0 0x7fb48c0a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49274e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c09d320 con 0x7fb48c09b5b0 2023-01-25T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49174c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c09b5b0 0x7fb48c0a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.385+0000 7fb49174c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c09b5b0 0x7fb48c0a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38610/0 (socket says 172.21.15.117:38610) 2023-01-25T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb49174c700 1 -- 172.21.15.117:0/3098058758 learned_addr learned my addr 172.21.15.117:0/3098058758 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-01-25T06:49:34.396 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb49174c700 1 -- 172.21.15.117:0/3098058758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c0094b0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb49174c700 1 --2- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c09b5b0 0x7fb48c0a13b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c450 tx=0x7fb48800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb481ffb700 1 -- 172.21.15.117:0/3098058758 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb488005b90 con 0x7fb48c09b5b0 2023-01-25T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb481ffb700 1 -- 172.21.15.117:0/3098058758 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb488005cf0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c133f40 con 0x7fb48c09b5b0 2023-01-25T06:49:34.397 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb481ffb700 1 -- 172.21.15.117:0/3098058758 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4880070d0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c133db0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb481ffb700 1 -- 172.21.15.117:0/3098058758 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 41514+0+0 (secure 0 0 0) 0x7fb488019030 con 0x7fb48c09b5b0 2023-01-25T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c1340d0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.398 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb481ffb700 1 --2- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb470033d80 0x7fb470036230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb481ffb700 1 -- 172.21.15.117:0/3098058758 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3406+0+0 (secure 0 0 0) 0x7fb4880447a0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb490f4b700 1 --2- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb470033d80 0x7fb470036230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T06:49:34.399 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.389+0000 7fb490f4b700 1 --2- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb470033d80 0x7fb470036230 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0090b0 tx=0x7fb47c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T06:49:34.401 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.397+0000 7fb481ffb700 1 -- 172.21.15.117:0/3098058758 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+168067 (secure 0 0 0) 0x7fb4880198b0 con 0x7fb48c09b5b0 2023-01-25T06:49:34.560 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.557+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 --> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb48c12c140 con 0x7fb470033d80 2023-01-25T06:49:34.575 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.569+0000 7fb481ffb700 1 -- 172.21.15.117:0/3098058758 <== mgr.4101 v2:172.21.15.117:6824/25572 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+255635 (secure 0 0 0) 0x7fb48c12c140 con 0x7fb470033d80 2023-01-25T06:49:34.639 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-01-25T06:49:34.643 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb470033d80 msgr2=0x7fb470036230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:34.643 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 --2- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb470033d80 0x7fb470036230 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0090b0 tx=0x7fb47c008030 comp rx=0 tx=0).stop 2023-01-25T06:49:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c09b5b0 msgr2=0x7fb48c0a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T06:49:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 --2- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c09b5b0 0x7fb48c0a13b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c450 tx=0x7fb48800c810 comp rx=0 tx=0).stop 2023-01-25T06:49:34.644 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 shutdown_connections 2023-01-25T06:49:34.645 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 --2- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:6824/25572,v1:172.21.15.117:6825/25572] conn(0x7fb470033d80 0x7fb470036230 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:34.645 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 --2- 172.21.15.117:0/3098058758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb48c09b5b0 0x7fb48c0a13b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T06:49:34.645 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 >> 172.21.15.117:0/3098058758 conn(0x7fb48c06de50 msgr2=0x7fb48c12edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T06:49:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 shutdown_connections 2023-01-25T06:49:34.646 INFO:teuthology.orchestra.run.smithi117.stderr:2023-01-25T06:49:34.573+0000 7fb49274e700 1 -- 172.21.15.117:0/3098058758 wait complete. 2023-01-25T06:49:34.647 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2023-01-25T06:49:34.647 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-01-25T06:49:33.695106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":15572,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":15168,"kb_avail":314557228,"statfs":{"total":322122547200,"available":322106601472,"internally_reserved":0,"allocated":389120,"data_stored":98636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15532032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097401+0000","last_change":"2023-01-25T06:49:31.116814+0000","last_active":"2023-01-25T06:49:32.097401+0000","last_peered":"2023-01-25T06:49:32.097401+0000","last_clean":"2023-01-25T06:49:32.097401+0000","last_became_active":"2023-01-25T06:49:31.116493+0000","last_became_peered":"2023-01-25T06:49:31.116493+0000","last_unstale":"2023-01-25T06:49:32.097401+0000","last_undegraded":"2023-01-25T06:49:32.097401+0000","last_fullsized":"2023-01-25T06:49:32.097401+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:25:40.674339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097628+0000","last_change":"2023-01-25T06:49:31.118198+0000","last_active":"2023-01-25T06:49:32.097628+0000","last_peered":"2023-01-25T06:49:32.097628+0000","last_clean":"2023-01-25T06:49:32.097628+0000","last_became_active":"2023-01-25T06:49:31.117967+0000","last_became_peered":"2023-01-25T06:49:31.117967+0000","last_unstale":"2023-01-25T06:49:32.097628+0000","last_undegraded":"2023-01-25T06:49:32.097628+0000","last_fullsized":"2023-01-25T06:49:32.097628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:36:34.519717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096003+0000","last_change":"2023-01-25T06:49:31.110220+0000","last_active":"2023-01-25T06:49:32.096003+0000","last_peered":"2023-01-25T06:49:32.096003+0000","last_clean":"2023-01-25T06:49:32.096003+0000","last_became_active":"2023-01-25T06:49:31.109897+0000","last_became_peered":"2023-01-25T06:49:31.109897+0000","last_unstale":"2023-01-25T06:49:32.096003+0000","last_undegraded":"2023-01-25T06:49:32.096003+0000","last_fullsized":"2023-01-25T06:49:32.096003+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T08:03:02.699337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097873+0000","last_change":"2023-01-25T06:49:31.117725+0000","last_active":"2023-01-25T06:49:32.097873+0000","last_peered":"2023-01-25T06:49:32.097873+0000","last_clean":"2023-01-25T06:49:32.097873+0000","last_became_active":"2023-01-25T06:49:31.117358+0000","last_became_peered":"2023-01-25T06:49:31.117358+0000","last_unstale":"2023-01-25T06:49:32.097873+0000","last_undegraded":"2023-01-25T06:49:32.097873+0000","last_fullsized":"2023-01-25T06:49:32.097873+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:59:52.778738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096535+0000","last_change":"2023-01-25T06:49:31.110964+0000","last_active":"2023-01-25T06:49:32.096535+0000","last_peered":"2023-01-25T06:49:32.096535+0000","last_clean":"2023-01-25T06:49:32.096535+0000","last_became_active":"2023-01-25T06:49:31.110055+0000","last_became_peered":"2023-01-25T06:49:31.110055+0000","last_unstale":"2023-01-25T06:49:32.096535+0000","last_undegraded":"2023-01-25T06:49:32.096535+0000","last_fullsized":"2023-01-25T06:49:32.096535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T07:13:32.029208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.097089+0000","last_change":"2023-01-25T06:49:31.118370+0000","last_active":"2023-01-25T06:49:32.097089+0000","last_peered":"2023-01-25T06:49:32.097089+0000","last_clean":"2023-01-25T06:49:32.097089+0000","last_became_active":"2023-01-25T06:49:31.117918+0000","last_became_peered":"2023-01-25T06:49:31.117918+0000","last_unstale":"2023-01-25T06:49:32.097089+0000","last_undegraded":"2023-01-25T06:49:32.097089+0000","last_fullsized":"2023-01-25T06:49:32.097089+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T11:23:37.727304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:30.710431+0000","last_change":"2023-01-25T06:49:30.710431+0000","last_active":"2023-01-25T06:49:30.710431+0000","last_peered":"2023-01-25T06:49:30.710431+0000","last_clean":"2023-01-25T06:49:30.710431+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:30.710431+0000","last_undegraded":"2023-01-25T06:49:30.710431+0000","last_fullsized":"2023-01-25T06:49:30.710431+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.096453+0000","last_change":"2023-01-25T06:49:31.110594+0000","last_active":"2023-01-25T06:49:32.096453+0000","last_peered":"2023-01-25T06:49:32.096453+0000","last_clean":"2023-01-25T06:49:32.096453+0000","last_became_active":"2023-01-25T06:49:31.110364+0000","last_became_peered":"2023-01-25T06:49:31.110364+0000","last_unstale":"2023-01-25T06:49:32.096453+0000","last_undegraded":"2023-01-25T06:49:32.096453+0000","last_fullsized":"2023-01-25T06:49:32.096453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:25:28.364287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-01-25T06:49:29.703081+0000","last_change":"2023-01-25T06:49:29.703081+0000","last_active":"2023-01-25T06:49:29.703081+0000","last_peered":"2023-01-25T06:49:29.703081+0000","last_clean":"2023-01-25T06:49:29.703081+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-01-25T06:49:29.703081+0000","last_undegraded":"2023-01-25T06:49:29.703081+0000","last_fullsized":"2023-01-25T06:49:29.703081+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:29.703081+0000","last_clean_scrub_stamp":"2023-01-25T06:49:29.703081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:49:32.095669+0000","last_change":"2023-01-25T06:49:31.114813+0000","last_active":"2023-01-25T06:49:32.095669+0000","last_peered":"2023-01-25T06:49:32.095669+0000","last_clean":"2023-01-25T06:49:32.095669+0000","last_became_active":"2023-01-25T06:49:31.109639+0000","last_became_peered":"2023-01-25T06:49:31.109639+0000","last_unstale":"2023-01-25T06:49:32.095669+0000","last_undegraded":"2023-01-25T06:49:32.095669+0000","last_fullsized":"2023-01-25T06:49:32.095669+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:49:30.710431+0000","last_clean_scrub_stamp":"2023-01-25T06:49:30.710431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T17:00:39.780533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_se